Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 4,
 'device': device(type='cuda', index=1),
 'dropout': 0.25,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1500,
 'ffn_num_layers': 1,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1500,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.061553
Epoch 1
Validation binary_cross_entropy = 1.952785
Epoch 2
Validation binary_cross_entropy = 1.050629
Epoch 3
Validation binary_cross_entropy = 2.124702
Epoch 4
Loss = 2.1540e+00, PNorm = 57.7633, GNorm = 32.0461, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.744349
Epoch 5
Validation binary_cross_entropy = 0.738668
Epoch 6
Validation binary_cross_entropy = 1.215603
Epoch 7
Validation binary_cross_entropy = 0.698267
Epoch 8
Validation binary_cross_entropy = 0.402070
Epoch 9
Loss = 1.2927e+00, PNorm = 57.9838, GNorm = 14.9949, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.587233
Epoch 10
Validation binary_cross_entropy = 0.737592
Epoch 11
Validation binary_cross_entropy = 0.666846
Epoch 12
Validation binary_cross_entropy = 0.649297
Epoch 13
Validation binary_cross_entropy = 0.659452
Epoch 14
Loss = 9.0531e-01, PNorm = 58.2608, GNorm = 11.2939, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.455253
Epoch 15
Validation binary_cross_entropy = 0.769867
Epoch 16
Validation binary_cross_entropy = 1.209972
Epoch 17
Validation binary_cross_entropy = 0.317010
Epoch 18
Validation binary_cross_entropy = 0.505908
Epoch 19
Loss = 1.0918e+00, PNorm = 58.4891, GNorm = 20.6620, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.700164
Epoch 20
Validation binary_cross_entropy = 0.594571
Epoch 21
Validation binary_cross_entropy = 0.538045
Epoch 22
Validation binary_cross_entropy = 0.437080
Epoch 23
Validation binary_cross_entropy = 0.525593
Epoch 24
Loss = 1.1784e+00, PNorm = 58.7068, GNorm = 32.0006, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.867468
Epoch 25
Validation binary_cross_entropy = 0.665526
Epoch 26
Validation binary_cross_entropy = 0.417536
Epoch 27
Validation binary_cross_entropy = 0.530122
Epoch 28
Validation binary_cross_entropy = 0.749791
Epoch 29
Loss = 9.9092e-01, PNorm = 58.9069, GNorm = 11.4036, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.380775
Epoch 30
Validation binary_cross_entropy = 0.312273
Epoch 31
Validation binary_cross_entropy = 0.345015
Epoch 32
Validation binary_cross_entropy = 0.613400
Epoch 33
Validation binary_cross_entropy = 0.671160
Epoch 34
Loss = 9.1311e-01, PNorm = 59.1042, GNorm = 17.8517, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.488269
Epoch 35
Validation binary_cross_entropy = 0.341690
Epoch 36
Validation binary_cross_entropy = 0.349789
Epoch 37
Validation binary_cross_entropy = 0.456931
Epoch 38
Validation binary_cross_entropy = 0.845190
Epoch 39
Loss = 9.9146e-01, PNorm = 59.3349, GNorm = 25.3176, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.710545
Epoch 40
Validation binary_cross_entropy = 0.406723
Epoch 41
Validation binary_cross_entropy = 0.335951
Epoch 42
Validation binary_cross_entropy = 0.343974
Epoch 43
Validation binary_cross_entropy = 0.505171
Epoch 44
Loss = 8.6027e-01, PNorm = 59.5636, GNorm = 12.5379, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.546793
Epoch 45
Validation binary_cross_entropy = 0.339766
Epoch 46
Validation binary_cross_entropy = 0.349148
Epoch 47
Validation binary_cross_entropy = 0.524975
Epoch 48
Validation binary_cross_entropy = 0.482815
Epoch 49
Loss = 5.8681e-01, PNorm = 59.8246, GNorm = 17.6500, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.418338
Epoch 50
Validation binary_cross_entropy = 0.411886
Epoch 51
Validation binary_cross_entropy = 0.397754
Epoch 52
Validation binary_cross_entropy = 0.465090
Epoch 53
Validation binary_cross_entropy = 0.445153
Epoch 54
Loss = 6.1400e-01, PNorm = 60.1279, GNorm = 13.7091, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.355108
Epoch 55
Validation binary_cross_entropy = 0.393441
Epoch 56
Validation binary_cross_entropy = 0.495662
Epoch 57
Validation binary_cross_entropy = 0.403494
Epoch 58
Validation binary_cross_entropy = 0.461513
Epoch 59
Loss = 7.1174e-01, PNorm = 60.4356, GNorm = 15.1247, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.606208
Epoch 60
Validation binary_cross_entropy = 0.378991
Epoch 61
Validation binary_cross_entropy = 0.384270
Epoch 62
Validation binary_cross_entropy = 0.605976
Epoch 63
Validation binary_cross_entropy = 0.447453
Epoch 64
Loss = 6.6218e-01, PNorm = 60.7950, GNorm = 35.2388, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.379973
Epoch 65
Validation binary_cross_entropy = 0.405083
Epoch 66
Validation binary_cross_entropy = 0.548722
Epoch 67
Validation binary_cross_entropy = 0.711087
Epoch 68
Validation binary_cross_entropy = 0.510898
Epoch 69
Loss = 5.0154e-01, PNorm = 61.1446, GNorm = 10.7157, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.506474
Epoch 70
Validation binary_cross_entropy = 0.524359
Epoch 71
Validation binary_cross_entropy = 0.662071
Epoch 72
Validation binary_cross_entropy = 0.609118
Epoch 73
Validation binary_cross_entropy = 0.471336
Epoch 74
Loss = 4.3868e-01, PNorm = 61.5073, GNorm = 20.2137, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.585563
Epoch 75
Validation binary_cross_entropy = 0.482240
Epoch 76
Validation binary_cross_entropy = 0.441891
Epoch 77
Validation binary_cross_entropy = 0.508779
Epoch 78
Validation binary_cross_entropy = 0.767668
Epoch 79
Loss = 2.6512e-01, PNorm = 61.9173, GNorm = 13.2676, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.867301
Epoch 80
Validation binary_cross_entropy = 0.786108
Epoch 81
Validation binary_cross_entropy = 0.736662
Epoch 82
Validation binary_cross_entropy = 0.622951
Epoch 83
Validation binary_cross_entropy = 0.458499
Epoch 84
Loss = 4.0875e-01, PNorm = 62.3033, GNorm = 26.2294, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.437068
Epoch 85
Validation binary_cross_entropy = 0.449288
Epoch 86
Validation binary_cross_entropy = 0.468526
Epoch 87
Validation binary_cross_entropy = 0.462047
Epoch 88
Validation binary_cross_entropy = 0.503835
Epoch 89
Loss = 3.8704e-01, PNorm = 62.6562, GNorm = 9.0712, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.584139
Epoch 90
Validation binary_cross_entropy = 0.614106
Epoch 91
Validation binary_cross_entropy = 0.578569
Epoch 92
Validation binary_cross_entropy = 0.570822
Epoch 93
Validation binary_cross_entropy = 0.582471
Epoch 94
Loss = 3.0736e-01, PNorm = 63.0083, GNorm = 6.5710, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.488528
Epoch 95
Validation binary_cross_entropy = 0.513574
Epoch 96
Validation binary_cross_entropy = 0.496926
Epoch 97
Validation binary_cross_entropy = 0.539530
Epoch 98
Validation binary_cross_entropy = 0.609104
Epoch 99
Loss = 1.6333e-01, PNorm = 63.3153, GNorm = 4.1334, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.635701
Epoch 100
Validation binary_cross_entropy = 0.661672
Epoch 101
Validation binary_cross_entropy = 0.678205
Epoch 102
Validation binary_cross_entropy = 0.701458
Epoch 103
Validation binary_cross_entropy = 0.778856
Epoch 104
Loss = 1.4161e-01, PNorm = 63.5846, GNorm = 11.6006, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.746761
Epoch 105
Validation binary_cross_entropy = 0.713358
Epoch 106
Validation binary_cross_entropy = 0.690224
Epoch 107
Validation binary_cross_entropy = 0.640898
Epoch 108
Validation binary_cross_entropy = 0.636962
Epoch 109
Loss = 2.1771e-01, PNorm = 63.8539, GNorm = 9.5335, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.679341
Epoch 110
Validation binary_cross_entropy = 0.720054
Epoch 111
Validation binary_cross_entropy = 0.746858
Epoch 112
Validation binary_cross_entropy = 0.773381
Epoch 113
Validation binary_cross_entropy = 0.828059
Epoch 114
Loss = 1.1643e-01, PNorm = 64.1085, GNorm = 7.7137, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.961308
Epoch 115
Validation binary_cross_entropy = 1.047400
Epoch 116
Validation binary_cross_entropy = 1.118915
Epoch 117
Validation binary_cross_entropy = 1.090369
Epoch 118
Validation binary_cross_entropy = 1.085397
Epoch 119
Loss = 3.3319e-01, PNorm = 64.3656, GNorm = 23.4620, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.834796
Epoch 120
Validation binary_cross_entropy = 0.860020
Epoch 121
Validation binary_cross_entropy = 0.727581
Epoch 122
Validation binary_cross_entropy = 0.760414
Epoch 123
Validation binary_cross_entropy = 0.714580
Epoch 124
Loss = 3.1498e-01, PNorm = 64.6337, GNorm = 8.3669, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.761463
Epoch 125
Validation binary_cross_entropy = 0.788403
Epoch 126
Validation binary_cross_entropy = 0.780249
Epoch 127
Validation binary_cross_entropy = 0.804398
Epoch 128
Validation binary_cross_entropy = 0.802950
Epoch 129
Loss = 1.2922e-01, PNorm = 64.8921, GNorm = 11.0637, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.825133
Epoch 130
Validation binary_cross_entropy = 0.847513
Epoch 131
Validation binary_cross_entropy = 0.886162
Epoch 132
Validation binary_cross_entropy = 0.948147
Epoch 133
Validation binary_cross_entropy = 1.021853
Epoch 134
Loss = 1.9181e-01, PNorm = 65.1004, GNorm = 11.5495, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.961341
Epoch 135
Validation binary_cross_entropy = 0.861600
Epoch 136
Validation binary_cross_entropy = 0.836084
Epoch 137
Validation binary_cross_entropy = 0.756904
Epoch 138
Validation binary_cross_entropy = 0.749279
Epoch 139
Loss = 1.1432e-01, PNorm = 65.3251, GNorm = 9.7464, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.752994
Epoch 140
Validation binary_cross_entropy = 0.770333
Epoch 141
Validation binary_cross_entropy = 0.814085
Epoch 142
Validation binary_cross_entropy = 0.869581
Epoch 143
Validation binary_cross_entropy = 0.930943
Epoch 144
Loss = 1.0793e-01, PNorm = 65.5290, GNorm = 2.8579, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.009385
Epoch 145
Validation binary_cross_entropy = 1.049407
Epoch 146
Validation binary_cross_entropy = 1.154688
Epoch 147
Validation binary_cross_entropy = 1.185212
Epoch 148
Validation binary_cross_entropy = 1.231274
Epoch 149
Loss = 8.5208e-02, PNorm = 65.6908, GNorm = 6.4435, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.327151
Epoch 150
Validation binary_cross_entropy = 1.329159
Epoch 151
Validation binary_cross_entropy = 1.274061
Epoch 152
Validation binary_cross_entropy = 1.394163
Epoch 153
Validation binary_cross_entropy = 1.190041
Epoch 154
Loss = 1.9817e-01, PNorm = 65.8973, GNorm = 5.3860, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.005758
Epoch 155
Validation binary_cross_entropy = 1.041627
Epoch 156
Validation binary_cross_entropy = 0.900171
Epoch 157
Validation binary_cross_entropy = 0.799618
Epoch 158
Validation binary_cross_entropy = 0.803858
Epoch 159
Loss = 1.4852e-01, PNorm = 66.1649, GNorm = 4.5059, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.809249
Epoch 160
Validation binary_cross_entropy = 0.863945
Epoch 161
Validation binary_cross_entropy = 0.934167
Epoch 162
Validation binary_cross_entropy = 0.856041
Epoch 163
Validation binary_cross_entropy = 0.855914
Epoch 164
Loss = 2.4280e-01, PNorm = 66.3714, GNorm = 9.5079, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.911471
Epoch 165
Validation binary_cross_entropy = 0.873227
Epoch 166
Validation binary_cross_entropy = 0.883344
Epoch 167
Validation binary_cross_entropy = 0.919510
Epoch 168
Validation binary_cross_entropy = 0.988197
Epoch 169
Loss = 8.0848e-02, PNorm = 66.5346, GNorm = 5.3623, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.052581
Epoch 170
Validation binary_cross_entropy = 1.094019
Epoch 171
Validation binary_cross_entropy = 1.097405
Epoch 172
Validation binary_cross_entropy = 1.118991
Epoch 173
Validation binary_cross_entropy = 1.098918
Epoch 174
Loss = 6.7718e-02, PNorm = 66.6618, GNorm = 0.6019, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.084204
Epoch 175
Validation binary_cross_entropy = 1.077584
Epoch 176
Validation binary_cross_entropy = 1.088872
Epoch 177
Validation binary_cross_entropy = 1.125327
Epoch 178
Validation binary_cross_entropy = 1.087357
Epoch 179
Loss = 3.2857e-02, PNorm = 66.7634, GNorm = 0.7978, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.025790
Epoch 180
Validation binary_cross_entropy = 1.017833
Epoch 181
Validation binary_cross_entropy = 1.040827
Epoch 182
Validation binary_cross_entropy = 1.118675
Epoch 183
Validation binary_cross_entropy = 1.167785
Epoch 184
Loss = 2.7232e-02, PNorm = 66.8720, GNorm = 1.0809, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.154688
Epoch 185
Validation binary_cross_entropy = 1.145995
Epoch 186
Validation binary_cross_entropy = 1.173096
Epoch 187
Validation binary_cross_entropy = 1.189301
Epoch 188
Validation binary_cross_entropy = 1.233284
Epoch 189
Loss = 4.9702e-03, PNorm = 66.9689, GNorm = 0.2692, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.287100
Epoch 190
Validation binary_cross_entropy = 1.329991
Epoch 191
Validation binary_cross_entropy = 1.404071
Epoch 192
Validation binary_cross_entropy = 1.373351
Epoch 193
Validation binary_cross_entropy = 1.371946
Epoch 194
Loss = 6.7084e-02, PNorm = 67.0366, GNorm = 13.1180, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.416352
Epoch 195
Validation binary_cross_entropy = 1.464396
Epoch 196
Validation binary_cross_entropy = 1.498411
Epoch 197
Validation binary_cross_entropy = 1.397762
Epoch 198
Validation binary_cross_entropy = 1.357775
Epoch 199
Loss = 2.4139e-02, PNorm = 67.1139, GNorm = 3.6044, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.375697
Epoch 200
Validation binary_cross_entropy = 1.459417
Epoch 201
Validation binary_cross_entropy = 1.494909
Epoch 202
Validation binary_cross_entropy = 1.362591
Epoch 203
Validation binary_cross_entropy = 1.295985
Epoch 204
Loss = 2.0431e-02, PNorm = 67.2233, GNorm = 1.2075, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.285933
Epoch 205
Validation binary_cross_entropy = 1.278333
Epoch 206
Validation binary_cross_entropy = 1.256160
Epoch 207
Validation binary_cross_entropy = 1.270942
Epoch 208
Validation binary_cross_entropy = 1.261908
Epoch 209
Loss = 4.4334e-02, PNorm = 67.3038, GNorm = 4.3882, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.236463
Epoch 210
Validation binary_cross_entropy = 1.181975
Epoch 211
Validation binary_cross_entropy = 1.180774
Epoch 212
Validation binary_cross_entropy = 1.202101
Epoch 213
Validation binary_cross_entropy = 1.278190
Epoch 214
Loss = 9.4949e-03, PNorm = 67.3959, GNorm = 4.3460, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.426372
Epoch 215
Validation binary_cross_entropy = 1.326964
Epoch 216
Validation binary_cross_entropy = 1.290657
Epoch 217
Validation binary_cross_entropy = 1.300100
Epoch 218
Validation binary_cross_entropy = 1.299936
Epoch 219
Loss = 1.5433e-02, PNorm = 67.5188, GNorm = 1.9754, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.330347
Epoch 220
Validation binary_cross_entropy = 1.314676
Epoch 221
Validation binary_cross_entropy = 1.258103
Epoch 222
Validation binary_cross_entropy = 1.281274
Epoch 223
Validation binary_cross_entropy = 1.310607
Epoch 224
Loss = 6.2279e-02, PNorm = 67.6391, GNorm = 8.8636, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.365894
Epoch 225
Validation binary_cross_entropy = 1.514454
Epoch 226
Validation binary_cross_entropy = 1.394214
Epoch 227
Validation binary_cross_entropy = 1.262898
Epoch 228
Validation binary_cross_entropy = 1.154104
Epoch 229
Loss = 2.3938e-01, PNorm = 67.7893, GNorm = 3.9730, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.121008
Epoch 230
Validation binary_cross_entropy = 1.226943
Epoch 231
Validation binary_cross_entropy = 1.112829
Epoch 232
Validation binary_cross_entropy = 1.018341
Epoch 233
Validation binary_cross_entropy = 1.070692
Epoch 234
Loss = 8.9671e-02, PNorm = 67.9865, GNorm = 6.0175, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.147239
Epoch 235
Validation binary_cross_entropy = 1.266346
Epoch 236
Validation binary_cross_entropy = 1.472893
Epoch 237
Validation binary_cross_entropy = 1.608551
Epoch 238
Validation binary_cross_entropy = 1.483447
Epoch 239
Loss = 7.7661e-02, PNorm = 68.1788, GNorm = 2.7950, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.445730
Epoch 240
Validation binary_cross_entropy = 1.443998
Epoch 241
Validation binary_cross_entropy = 1.449119
Epoch 242
Validation binary_cross_entropy = 1.484831
Epoch 243
Validation binary_cross_entropy = 1.419027
Epoch 244
Loss = 1.2378e-02, PNorm = 68.3270, GNorm = 0.9910, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.315883
Epoch 245
Validation binary_cross_entropy = 1.261539
Epoch 246
Validation binary_cross_entropy = 1.207861
Epoch 247
Validation binary_cross_entropy = 1.184383
Epoch 248
Validation binary_cross_entropy = 1.216102
Epoch 249
Loss = 1.0882e-02, PNorm = 68.4224, GNorm = 0.8202, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.268095
Epoch 250
Validation binary_cross_entropy = 1.294762
Epoch 251
Validation binary_cross_entropy = 1.308683
Epoch 252
Validation binary_cross_entropy = 1.341288
Epoch 253
Validation binary_cross_entropy = 1.381421
Epoch 254
Loss = 8.9101e-02, PNorm = 68.4966, GNorm = 13.7118, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.437708
Epoch 255
Validation binary_cross_entropy = 1.572229
Epoch 256
Validation binary_cross_entropy = 1.617351
Epoch 257
Validation binary_cross_entropy = 1.597434
Epoch 258
Validation binary_cross_entropy = 1.566726
Epoch 259
Loss = 3.3021e-02, PNorm = 68.5752, GNorm = 2.9547, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.573749
Epoch 260
Validation binary_cross_entropy = 1.580509
Epoch 261
Validation binary_cross_entropy = 1.590288
Epoch 262
Validation binary_cross_entropy = 1.579911
Epoch 263
Validation binary_cross_entropy = 1.537224
Epoch 264
Loss = 5.8328e-02, PNorm = 68.6296, GNorm = 4.2974, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.394686
Epoch 265
Validation binary_cross_entropy = 1.316741
Epoch 266
Validation binary_cross_entropy = 1.307949
Epoch 267
Validation binary_cross_entropy = 1.366349
Epoch 268
Validation binary_cross_entropy = 1.380627
Epoch 269
Loss = 2.0233e-02, PNorm = 68.7193, GNorm = 1.4460, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.353676
Epoch 270
Validation binary_cross_entropy = 1.375039
Epoch 271
Validation binary_cross_entropy = 1.411032
Epoch 272
Validation binary_cross_entropy = 1.454965
Epoch 273
Validation binary_cross_entropy = 1.517807
Epoch 274
Loss = 6.4130e-03, PNorm = 68.7916, GNorm = 0.2416, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.579183
Epoch 275
Validation binary_cross_entropy = 1.643004
Epoch 276
Validation binary_cross_entropy = 1.672865
Epoch 277
Validation binary_cross_entropy = 1.689023
Epoch 278
Validation binary_cross_entropy = 1.702638
Epoch 279
Loss = 4.7241e-03, PNorm = 68.8457, GNorm = 0.3568, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.708815
Epoch 280
Validation binary_cross_entropy = 1.706464
Epoch 281
Validation binary_cross_entropy = 1.726383
Epoch 282
Validation binary_cross_entropy = 1.722010
Epoch 283
Validation binary_cross_entropy = 1.698631
Epoch 284
Loss = 6.4526e-03, PNorm = 68.8802, GNorm = 0.1054, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.671343
Epoch 285
Validation binary_cross_entropy = 1.647997
Epoch 286
Validation binary_cross_entropy = 1.628414
Epoch 287
Validation binary_cross_entropy = 1.632970
Epoch 288
Validation binary_cross_entropy = 1.690309
Epoch 289
Loss = 1.2663e-02, PNorm = 68.9171, GNorm = 5.3466, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.724672
Epoch 290
Validation binary_cross_entropy = 1.699440
Epoch 291
Validation binary_cross_entropy = 1.650568
Epoch 292
Validation binary_cross_entropy = 1.643856
Epoch 293
Validation binary_cross_entropy = 1.646766
Epoch 294
Loss = 3.5077e-03, PNorm = 68.9499, GNorm = 0.7383, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.652238
Epoch 295
Validation binary_cross_entropy = 1.654742
Epoch 296
Validation binary_cross_entropy = 2.196722
Epoch 297
Validation binary_cross_entropy = 2.244299
Epoch 298
Validation binary_cross_entropy = 2.286349
Epoch 299
Loss = 1.0973e-01, PNorm = 68.9975, GNorm = 3.1736, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.859520
Model 0 best validation binary_cross_entropy = 0.312273 on epoch 30
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.263407
Ensemble test binary_cross_entropy = 0.263407
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.339485
Epoch 1
Validation binary_cross_entropy = 1.854031
Epoch 2
Validation binary_cross_entropy = 1.218056
Epoch 3
Validation binary_cross_entropy = 1.180431
Epoch 4
Loss = 2.2429e+00, PNorm = 57.7606, GNorm = 35.8793, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 1.068295
Epoch 5
Validation binary_cross_entropy = 1.033472
Epoch 6
Validation binary_cross_entropy = 1.590664
Epoch 7
Validation binary_cross_entropy = 0.994442
Epoch 8
Validation binary_cross_entropy = 0.419899
Epoch 9
Loss = 1.3871e+00, PNorm = 57.9572, GNorm = 15.5011, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.502540
Epoch 10
Validation binary_cross_entropy = 0.603389
Epoch 11
Validation binary_cross_entropy = 0.725885
Epoch 12
Validation binary_cross_entropy = 0.655782
Epoch 13
Validation binary_cross_entropy = 0.541790
Epoch 14
Loss = 1.5248e+00, PNorm = 58.2074, GNorm = 21.9866, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.507959
Epoch 15
Validation binary_cross_entropy = 0.484889
Epoch 16
Validation binary_cross_entropy = 0.676238
Epoch 17
Validation binary_cross_entropy = 0.691498
Epoch 18
Validation binary_cross_entropy = 0.491211
Epoch 19
Loss = 1.1285e+00, PNorm = 58.3972, GNorm = 23.8240, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.440535
Epoch 20
Validation binary_cross_entropy = 0.488225
Epoch 21
Validation binary_cross_entropy = 0.474498
Epoch 22
Validation binary_cross_entropy = 0.417951
Epoch 23
Validation binary_cross_entropy = 0.319991
Epoch 24
Loss = 1.2347e+00, PNorm = 58.5625, GNorm = 18.1356, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.352909
Epoch 25
Validation binary_cross_entropy = 0.839755
Epoch 26
Validation binary_cross_entropy = 0.700801
Epoch 27
Validation binary_cross_entropy = 0.359543
Epoch 28
Validation binary_cross_entropy = 0.313353
Epoch 29
Loss = 1.2726e+00, PNorm = 58.7465, GNorm = 31.4482, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.372182
Epoch 30
Validation binary_cross_entropy = 0.739291
Epoch 31
Validation binary_cross_entropy = 1.061247
Epoch 32
Validation binary_cross_entropy = 0.656972
Epoch 33
Validation binary_cross_entropy = 0.332573
Epoch 34
Loss = 7.9074e-01, PNorm = 58.9557, GNorm = 12.9517, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.293899
Epoch 35
Validation binary_cross_entropy = 0.320465
Epoch 36
Validation binary_cross_entropy = 0.439196
Epoch 37
Validation binary_cross_entropy = 0.508752
Epoch 38
Validation binary_cross_entropy = 0.492689
Epoch 39
Loss = 1.0625e+00, PNorm = 59.1232, GNorm = 12.4564, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.359857
Epoch 40
Validation binary_cross_entropy = 0.344454
Epoch 41
Validation binary_cross_entropy = 0.351628
Epoch 42
Validation binary_cross_entropy = 0.433630
Epoch 43
Validation binary_cross_entropy = 0.635121
Epoch 44
Loss = 6.7088e-01, PNorm = 59.2929, GNorm = 21.0745, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.566859
Epoch 45
Validation binary_cross_entropy = 0.393140
Epoch 46
Validation binary_cross_entropy = 0.365129
Epoch 47
Validation binary_cross_entropy = 0.443321
Epoch 48
Validation binary_cross_entropy = 0.427322
Epoch 49
Loss = 7.1915e-01, PNorm = 59.4946, GNorm = 13.3072, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.425752
Epoch 50
Validation binary_cross_entropy = 0.432568
Epoch 51
Validation binary_cross_entropy = 0.530018
Epoch 52
Validation binary_cross_entropy = 0.590296
Epoch 53
Validation binary_cross_entropy = 0.419883
Epoch 54
Loss = 7.0620e-01, PNorm = 59.7333, GNorm = 14.0341, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.415285
Epoch 55
Validation binary_cross_entropy = 0.691182
Epoch 56
Validation binary_cross_entropy = 0.438105
Epoch 57
Validation binary_cross_entropy = 0.397130
Epoch 58
Validation binary_cross_entropy = 0.505673
Epoch 59
Loss = 5.8548e-01, PNorm = 59.9939, GNorm = 29.0895, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.481111
Epoch 60
Validation binary_cross_entropy = 0.317228
Epoch 61
Validation binary_cross_entropy = 0.319925
Epoch 62
Validation binary_cross_entropy = 0.342335
Epoch 63
Validation binary_cross_entropy = 0.582637
Epoch 64
Loss = 6.6719e-01, PNorm = 60.2985, GNorm = 12.4475, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.715268
Epoch 65
Validation binary_cross_entropy = 0.522547
Epoch 66
Validation binary_cross_entropy = 0.490482
Epoch 67
Validation binary_cross_entropy = 0.555591
Epoch 68
Validation binary_cross_entropy = 0.641958
Epoch 69
Loss = 4.9615e-01, PNorm = 60.6463, GNorm = 13.7428, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.600739
Epoch 70
Validation binary_cross_entropy = 0.550114
Epoch 71
Validation binary_cross_entropy = 0.646795
Epoch 72
Validation binary_cross_entropy = 0.686020
Epoch 73
Validation binary_cross_entropy = 0.513838
Epoch 74
Loss = 4.3302e-01, PNorm = 60.9895, GNorm = 21.3796, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.463010
Epoch 75
Validation binary_cross_entropy = 0.507801
Epoch 76
Validation binary_cross_entropy = 0.510739
Epoch 77
Validation binary_cross_entropy = 0.444570
Epoch 78
Validation binary_cross_entropy = 0.477863
Epoch 79
Loss = 4.0866e-01, PNorm = 61.3233, GNorm = 18.1695, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.493316
Epoch 80
Validation binary_cross_entropy = 0.552882
Epoch 81
Validation binary_cross_entropy = 0.809188
Epoch 82
Validation binary_cross_entropy = 0.711164
Epoch 83
Validation binary_cross_entropy = 0.713060
Epoch 84
Loss = 3.8272e-01, PNorm = 61.6955, GNorm = 15.4502, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.529417
Epoch 85
Validation binary_cross_entropy = 0.484935
Epoch 86
Validation binary_cross_entropy = 0.522296
Epoch 87
Validation binary_cross_entropy = 0.512975
Epoch 88
Validation binary_cross_entropy = 0.505853
Epoch 89
Loss = 3.5355e-01, PNorm = 62.0365, GNorm = 9.0484, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.553218
Epoch 90
Validation binary_cross_entropy = 0.773711
Epoch 91
Validation binary_cross_entropy = 0.677538
Epoch 92
Validation binary_cross_entropy = 0.695501
Epoch 93
Validation binary_cross_entropy = 1.032438
Epoch 94
Loss = 3.5217e-01, PNorm = 62.3599, GNorm = 9.3240, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.574054
Epoch 95
Validation binary_cross_entropy = 0.542997
Epoch 96
Validation binary_cross_entropy = 0.576566
Epoch 97
Validation binary_cross_entropy = 0.720794
Epoch 98
Validation binary_cross_entropy = 0.606560
Epoch 99
Loss = 1.2697e-01, PNorm = 62.7113, GNorm = 9.4862, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.602100
Epoch 100
Validation binary_cross_entropy = 0.638334
Epoch 101
Validation binary_cross_entropy = 0.848274
Epoch 102
Validation binary_cross_entropy = 0.667000
Epoch 103
Validation binary_cross_entropy = 0.622315
Epoch 104
Loss = 2.9601e-01, PNorm = 63.0049, GNorm = 8.1230, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.612616
Epoch 105
Validation binary_cross_entropy = 0.609973
Epoch 106
Validation binary_cross_entropy = 0.667015
Epoch 107
Validation binary_cross_entropy = 0.659202
Epoch 108
Validation binary_cross_entropy = 0.698071
Epoch 109
Loss = 1.1250e-01, PNorm = 63.2624, GNorm = 4.4185, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.776752
Epoch 110
Validation binary_cross_entropy = 0.846532
Epoch 111
Validation binary_cross_entropy = 0.838047
Epoch 112
Validation binary_cross_entropy = 0.861981
Epoch 113
Validation binary_cross_entropy = 0.812640
Epoch 114
Loss = 1.9424e-01, PNorm = 63.5077, GNorm = 5.5802, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.841662
Epoch 115
Validation binary_cross_entropy = 0.976587
Epoch 116
Validation binary_cross_entropy = 0.846768
Epoch 117
Validation binary_cross_entropy = 0.870266
Epoch 118
Validation binary_cross_entropy = 0.865046
Epoch 119
Loss = 1.8943e-01, PNorm = 63.7482, GNorm = 10.5309, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.888429
Epoch 120
Validation binary_cross_entropy = 0.860999
Epoch 121
Validation binary_cross_entropy = 0.787124
Epoch 122
Validation binary_cross_entropy = 0.755563
Epoch 123
Validation binary_cross_entropy = 0.749097
Epoch 124
Loss = 3.0255e-01, PNorm = 63.9862, GNorm = 13.2140, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.752231
Epoch 125
Validation binary_cross_entropy = 0.808774
Epoch 126
Validation binary_cross_entropy = 0.790516
Epoch 127
Validation binary_cross_entropy = 0.795058
Epoch 128
Validation binary_cross_entropy = 0.804224
Epoch 129
Loss = 1.6966e-01, PNorm = 64.2345, GNorm = 12.6701, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.776750
Epoch 130
Validation binary_cross_entropy = 0.814930
Epoch 131
Validation binary_cross_entropy = 0.846077
Epoch 132
Validation binary_cross_entropy = 0.850412
Epoch 133
Validation binary_cross_entropy = 0.898119
Epoch 134
Loss = 8.5129e-02, PNorm = 64.4576, GNorm = 1.7812, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.949425
Epoch 135
Validation binary_cross_entropy = 0.951572
Epoch 136
Validation binary_cross_entropy = 0.967951
Epoch 137
Validation binary_cross_entropy = 0.987959
Epoch 138
Validation binary_cross_entropy = 1.001696
Epoch 139
Loss = 1.0371e-01, PNorm = 64.6389, GNorm = 0.3430, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.066887
Epoch 140
Validation binary_cross_entropy = 1.024058
Epoch 141
Validation binary_cross_entropy = 0.991621
Epoch 142
Validation binary_cross_entropy = 0.996448
Epoch 143
Validation binary_cross_entropy = 0.976274
Epoch 144
Loss = 2.3437e-01, PNorm = 64.8149, GNorm = 18.5797, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.990234
Epoch 145
Validation binary_cross_entropy = 1.130836
Epoch 146
Validation binary_cross_entropy = 1.101826
Epoch 147
Validation binary_cross_entropy = 1.152608
Epoch 148
Validation binary_cross_entropy = 1.039604
Epoch 149
Loss = 1.0198e-01, PNorm = 65.0086, GNorm = 6.4215, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.927079
Epoch 150
Validation binary_cross_entropy = 0.919150
Epoch 151
Validation binary_cross_entropy = 0.852084
Epoch 152
Validation binary_cross_entropy = 0.838422
Epoch 153
Validation binary_cross_entropy = 0.837203
Epoch 154
Loss = 5.3468e-02, PNorm = 65.1556, GNorm = 3.0102, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.860102
Epoch 155
Validation binary_cross_entropy = 0.948747
Epoch 156
Validation binary_cross_entropy = 1.008922
Epoch 157
Validation binary_cross_entropy = 1.099974
Epoch 158
Validation binary_cross_entropy = 1.190658
Epoch 159
Loss = 1.2278e-01, PNorm = 65.2991, GNorm = 0.8423, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.140012
Epoch 160
Validation binary_cross_entropy = 1.108611
Epoch 161
Validation binary_cross_entropy = 1.140134
Epoch 162
Validation binary_cross_entropy = 1.171345
Epoch 163
Validation binary_cross_entropy = 1.089684
Epoch 164
Loss = 1.5232e-01, PNorm = 65.4330, GNorm = 8.8023, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.075616
Epoch 165
Validation binary_cross_entropy = 1.082148
Epoch 166
Validation binary_cross_entropy = 1.145357
Epoch 167
Validation binary_cross_entropy = 1.153360
Epoch 168
Validation binary_cross_entropy = 1.178217
Epoch 169
Loss = 3.0758e-02, PNorm = 65.5741, GNorm = 0.4263, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.217415
Epoch 170
Validation binary_cross_entropy = 1.236331
Epoch 171
Validation binary_cross_entropy = 1.244348
Epoch 172
Validation binary_cross_entropy = 1.209891
Epoch 173
Validation binary_cross_entropy = 1.185066
Epoch 174
Loss = 7.8367e-02, PNorm = 65.7078, GNorm = 10.6334, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.129015
Epoch 175
Validation binary_cross_entropy = 1.117178
Epoch 176
Validation binary_cross_entropy = 1.396349
Epoch 177
Validation binary_cross_entropy = 1.461046
Epoch 178
Validation binary_cross_entropy = 1.159028
Epoch 179
Loss = 9.3051e-02, PNorm = 65.8430, GNorm = 3.6153, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.133127
Epoch 180
Validation binary_cross_entropy = 1.060454
Epoch 181
Validation binary_cross_entropy = 1.038465
Epoch 182
Validation binary_cross_entropy = 1.043237
Epoch 183
Validation binary_cross_entropy = 1.057166
Epoch 184
Loss = 1.4143e-01, PNorm = 65.9781, GNorm = 9.3052, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.079944
Epoch 185
Validation binary_cross_entropy = 1.113643
Epoch 186
Validation binary_cross_entropy = 1.140435
Epoch 187
Validation binary_cross_entropy = 1.181505
Epoch 188
Validation binary_cross_entropy = 1.237670
Epoch 189
Loss = 7.1247e-02, PNorm = 66.1043, GNorm = 3.7217, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.267857
Epoch 190
Validation binary_cross_entropy = 1.272758
Epoch 191
Validation binary_cross_entropy = 1.301592
Epoch 192
Validation binary_cross_entropy = 1.372101
Epoch 193
Validation binary_cross_entropy = 1.460853
Epoch 194
Loss = 4.6525e-02, PNorm = 66.2193, GNorm = 1.1561, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.465214
Epoch 195
Validation binary_cross_entropy = 1.492925
Epoch 196
Validation binary_cross_entropy = 1.480543
Epoch 197
Validation binary_cross_entropy = 1.426152
Epoch 198
Validation binary_cross_entropy = 1.344714
Epoch 199
Loss = 1.6530e-02, PNorm = 66.3231, GNorm = 1.8491, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.282807
Epoch 200
Validation binary_cross_entropy = 1.249387
Epoch 201
Validation binary_cross_entropy = 1.215031
Epoch 202
Validation binary_cross_entropy = 1.204763
Epoch 203
Validation binary_cross_entropy = 1.171524
Epoch 204
Loss = 1.2394e-02, PNorm = 66.3979, GNorm = 0.1295, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.189200
Epoch 205
Validation binary_cross_entropy = 1.209261
Epoch 206
Validation binary_cross_entropy = 1.188565
Epoch 207
Validation binary_cross_entropy = 1.173279
Epoch 208
Validation binary_cross_entropy = 1.139830
Epoch 209
Loss = 3.1627e-02, PNorm = 66.4862, GNorm = 3.5008, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.129839
Epoch 210
Validation binary_cross_entropy = 1.140572
Epoch 211
Validation binary_cross_entropy = 1.183038
Epoch 212
Validation binary_cross_entropy = 1.179639
Epoch 213
Validation binary_cross_entropy = 1.205601
Epoch 214
Loss = 1.4638e-02, PNorm = 66.5751, GNorm = 1.5627, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.236611
Epoch 215
Validation binary_cross_entropy = 1.261525
Epoch 216
Validation binary_cross_entropy = 1.294575
Epoch 217
Validation binary_cross_entropy = 1.292225
Epoch 218
Validation binary_cross_entropy = 1.296646
Epoch 219
Loss = 1.6865e-02, PNorm = 66.6599, GNorm = 2.5950, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.334654
Epoch 220
Validation binary_cross_entropy = 1.330981
Epoch 221
Validation binary_cross_entropy = 1.330667
Epoch 222
Validation binary_cross_entropy = 1.341895
Epoch 223
Validation binary_cross_entropy = 1.364345
Epoch 224
Loss = 2.6153e-02, PNorm = 66.7439, GNorm = 2.9295, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.372097
Epoch 225
Validation binary_cross_entropy = 1.352581
Epoch 226
Validation binary_cross_entropy = 1.325029
Epoch 227
Validation binary_cross_entropy = 1.319014
Epoch 228
Validation binary_cross_entropy = 1.346039
Epoch 229
Loss = 8.0460e-02, PNorm = 66.8215, GNorm = 3.2825, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.386625
Epoch 230
Validation binary_cross_entropy = 1.424115
Epoch 231
Validation binary_cross_entropy = 1.467827
Epoch 232
Validation binary_cross_entropy = 1.530850
Epoch 233
Validation binary_cross_entropy = 1.554179
Epoch 234
Loss = 1.1630e-02, PNorm = 66.9055, GNorm = 1.7620, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.579555
Epoch 235
Validation binary_cross_entropy = 1.590274
Epoch 236
Validation binary_cross_entropy = 1.595809
Epoch 237
Validation binary_cross_entropy = 1.479189
Epoch 238
Validation binary_cross_entropy = 1.996806
Epoch 239
Loss = 1.5307e-01, PNorm = 66.9821, GNorm = 8.5570, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.343333
Epoch 240
Validation binary_cross_entropy = 1.325777
Epoch 241
Validation binary_cross_entropy = 1.391724
Epoch 242
Validation binary_cross_entropy = 1.322150
Epoch 243
Validation binary_cross_entropy = 1.303110
Epoch 244
Loss = 3.2817e-02, PNorm = 67.1049, GNorm = 0.0933, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.325737
Epoch 245
Validation binary_cross_entropy = 1.376821
Epoch 246
Validation binary_cross_entropy = 1.402847
Epoch 247
Validation binary_cross_entropy = 1.275920
Epoch 248
Validation binary_cross_entropy = 1.228835
Epoch 249
Loss = 3.1002e-02, PNorm = 67.2661, GNorm = 7.3861, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.226982
Epoch 250
Validation binary_cross_entropy = 1.250801
Epoch 251
Validation binary_cross_entropy = 1.309578
Epoch 252
Validation binary_cross_entropy = 1.362834
Epoch 253
Validation binary_cross_entropy = 1.406519
Epoch 254
Loss = 1.1224e-02, PNorm = 67.4081, GNorm = 1.4853, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.460861
Epoch 255
Validation binary_cross_entropy = 1.488062
Epoch 256
Validation binary_cross_entropy = 1.502803
Epoch 257
Validation binary_cross_entropy = 1.516050
Epoch 258
Validation binary_cross_entropy = 1.531751
Epoch 259
Loss = 2.6041e-02, PNorm = 67.5056, GNorm = 2.2634, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.521185
Epoch 260
Validation binary_cross_entropy = 1.532331
Epoch 261
Validation binary_cross_entropy = 1.565672
Epoch 262
Validation binary_cross_entropy = 1.607632
Epoch 263
Validation binary_cross_entropy = 1.620420
Epoch 264
Loss = 2.5261e-02, PNorm = 67.5817, GNorm = 0.2900, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.635921
Epoch 265
Validation binary_cross_entropy = 1.652086
Epoch 266
Validation binary_cross_entropy = 1.672568
Epoch 267
Validation binary_cross_entropy = 1.699218
Epoch 268
Validation binary_cross_entropy = 1.718013
Epoch 269
Loss = 1.3750e-03, PNorm = 67.6360, GNorm = 0.2151, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.738511
Epoch 270
Validation binary_cross_entropy = 1.757797
Epoch 271
Validation binary_cross_entropy = 1.769904
Epoch 272
Validation binary_cross_entropy = 1.737451
Epoch 273
Validation binary_cross_entropy = 1.703029
Epoch 274
Loss = 1.2506e-02, PNorm = 67.6903, GNorm = 0.6334, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.675986
Epoch 275
Validation binary_cross_entropy = 1.667665
Epoch 276
Validation binary_cross_entropy = 1.682296
Epoch 277
Validation binary_cross_entropy = 1.747820
Epoch 278
Validation binary_cross_entropy = 1.819553
Epoch 279
Loss = 4.9460e-02, PNorm = 67.7443, GNorm = 3.1194, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.801135
Epoch 280
Validation binary_cross_entropy = 1.857543
Epoch 281
Validation binary_cross_entropy = 2.994698
Epoch 282
Validation binary_cross_entropy = 1.830309
Epoch 283
Validation binary_cross_entropy = 1.835732
Epoch 284
Loss = 5.2108e-02, PNorm = 67.8172, GNorm = 4.7270, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.866707
Epoch 285
Validation binary_cross_entropy = 1.782333
Epoch 286
Validation binary_cross_entropy = 1.778551
Epoch 287
Validation binary_cross_entropy = 1.716455
Epoch 288
Validation binary_cross_entropy = 1.643045
Epoch 289
Loss = 2.7483e-02, PNorm = 67.9002, GNorm = 2.4443, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.592185
Epoch 290
Validation binary_cross_entropy = 1.550300
Epoch 291
Validation binary_cross_entropy = 1.533349
Epoch 292
Validation binary_cross_entropy = 1.530621
Epoch 293
Validation binary_cross_entropy = 1.578161
Epoch 294
Loss = 6.7471e-03, PNorm = 67.9806, GNorm = 0.8747, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.716318
Epoch 295
Validation binary_cross_entropy = 1.804878
Epoch 296
Validation binary_cross_entropy = 1.689183
Epoch 297
Validation binary_cross_entropy = 2.215400
Epoch 298
Validation binary_cross_entropy = 2.226667
Epoch 299
Loss = 7.2313e-02, PNorm = 68.0526, GNorm = 6.2049, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.557773
Model 0 best validation binary_cross_entropy = 0.293899 on epoch 34
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.242477
Ensemble test binary_cross_entropy = 0.242477
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.045475
Epoch 1
Validation binary_cross_entropy = 1.989162
Epoch 2
Validation binary_cross_entropy = 1.065609
Epoch 3
Validation binary_cross_entropy = 0.969890
Epoch 4
Loss = 1.7929e+00, PNorm = 57.7629, GNorm = 20.0083, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 1.249619
Epoch 5
Validation binary_cross_entropy = 1.002144
Epoch 6
Validation binary_cross_entropy = 0.534127
Epoch 7
Validation binary_cross_entropy = 1.342515
Epoch 8
Validation binary_cross_entropy = 1.183787
Epoch 9
Loss = 1.2450e+00, PNorm = 57.9729, GNorm = 20.2900, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.866760
Epoch 10
Validation binary_cross_entropy = 0.598289
Epoch 11
Validation binary_cross_entropy = 0.590057
Epoch 12
Validation binary_cross_entropy = 0.757957
Epoch 13
Validation binary_cross_entropy = 0.689147
Epoch 14
Loss = 1.2574e+00, PNorm = 58.2210, GNorm = 13.5261, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.624327
Epoch 15
Validation binary_cross_entropy = 0.514494
Epoch 16
Validation binary_cross_entropy = 0.505543
Epoch 17
Validation binary_cross_entropy = 0.391196
Epoch 18
Validation binary_cross_entropy = 0.708532
Epoch 19
Loss = 1.0020e+00, PNorm = 58.4232, GNorm = 11.1242, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.627055
Epoch 20
Validation binary_cross_entropy = 0.504551
Epoch 21
Validation binary_cross_entropy = 0.379056
Epoch 22
Validation binary_cross_entropy = 0.402948
Epoch 23
Validation binary_cross_entropy = 0.633399
Epoch 24
Loss = 1.1292e+00, PNorm = 58.6018, GNorm = 12.6595, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.740951
Epoch 25
Validation binary_cross_entropy = 0.341062
Epoch 26
Validation binary_cross_entropy = 0.333874
Epoch 27
Validation binary_cross_entropy = 0.609169
Epoch 28
Validation binary_cross_entropy = 0.753132
Epoch 29
Loss = 1.1844e+00, PNorm = 58.7660, GNorm = 26.0199, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.497717
Epoch 30
Validation binary_cross_entropy = 0.352206
Epoch 31
Validation binary_cross_entropy = 0.318703
Epoch 32
Validation binary_cross_entropy = 0.322604
Epoch 33
Validation binary_cross_entropy = 0.439690
Epoch 34
Loss = 1.0375e+00, PNorm = 58.9515, GNorm = 29.6337, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.637099
Epoch 35
Validation binary_cross_entropy = 0.585590
Epoch 36
Validation binary_cross_entropy = 0.330724
Epoch 37
Validation binary_cross_entropy = 0.306648
Epoch 38
Validation binary_cross_entropy = 0.327692
Epoch 39
Loss = 9.2462e-01, PNorm = 59.1595, GNorm = 9.2885, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.503661
Epoch 40
Validation binary_cross_entropy = 0.534743
Epoch 41
Validation binary_cross_entropy = 0.429495
Epoch 42
Validation binary_cross_entropy = 0.394029
Epoch 43
Validation binary_cross_entropy = 0.378243
Epoch 44
Loss = 8.4840e-01, PNorm = 59.3551, GNorm = 16.6369, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.394593
Epoch 45
Validation binary_cross_entropy = 0.387513
Epoch 46
Validation binary_cross_entropy = 0.458116
Epoch 47
Validation binary_cross_entropy = 0.383321
Epoch 48
Validation binary_cross_entropy = 0.322890
Epoch 49
Loss = 6.3897e-01, PNorm = 59.5548, GNorm = 18.3903, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.354145
Epoch 50
Validation binary_cross_entropy = 0.429680
Epoch 51
Validation binary_cross_entropy = 0.431894
Epoch 52
Validation binary_cross_entropy = 0.443100
Epoch 53
Validation binary_cross_entropy = 0.465897
Epoch 54
Loss = 6.3038e-01, PNorm = 59.7988, GNorm = 8.8130, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.394894
Epoch 55
Validation binary_cross_entropy = 0.343645
Epoch 56
Validation binary_cross_entropy = 0.381380
Epoch 57
Validation binary_cross_entropy = 0.369017
Epoch 58
Validation binary_cross_entropy = 0.387076
Epoch 59
Loss = 6.3610e-01, PNorm = 60.0592, GNorm = 13.8372, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.535534
Epoch 60
Validation binary_cross_entropy = 0.383069
Epoch 61
Validation binary_cross_entropy = 0.548334
Epoch 62
Validation binary_cross_entropy = 0.465487
Epoch 63
Validation binary_cross_entropy = 0.491508
Epoch 64
Loss = 4.0693e-01, PNorm = 60.3514, GNorm = 7.4216, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.749312
Epoch 65
Validation binary_cross_entropy = 0.455078
Epoch 66
Validation binary_cross_entropy = 0.445659
Epoch 67
Validation binary_cross_entropy = 0.591431
Epoch 68
Validation binary_cross_entropy = 0.508465
Epoch 69
Loss = 3.2524e-01, PNorm = 60.7075, GNorm = 12.9549, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.524321
Epoch 70
Validation binary_cross_entropy = 0.423919
Epoch 71
Validation binary_cross_entropy = 0.538989
Epoch 72
Validation binary_cross_entropy = 0.510532
Epoch 73
Validation binary_cross_entropy = 0.474422
Epoch 74
Loss = 4.6506e-01, PNorm = 61.0660, GNorm = 7.4695, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 1.373329
Epoch 75
Validation binary_cross_entropy = 0.446140
Epoch 76
Validation binary_cross_entropy = 0.423364
Epoch 77
Validation binary_cross_entropy = 0.494343
Epoch 78
Validation binary_cross_entropy = 0.789182
Epoch 79
Loss = 9.0020e-01, PNorm = 61.4559, GNorm = 31.1382, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.388559
Epoch 80
Validation binary_cross_entropy = 0.370391
Epoch 81
Validation binary_cross_entropy = 0.357246
Epoch 82
Validation binary_cross_entropy = 0.387379
Epoch 83
Validation binary_cross_entropy = 0.518795
Epoch 84
Loss = 4.9938e-01, PNorm = 61.8363, GNorm = 13.1551, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.408169
Epoch 85
Validation binary_cross_entropy = 0.421078
Epoch 86
Validation binary_cross_entropy = 0.517710
Epoch 87
Validation binary_cross_entropy = 0.595527
Epoch 88
Validation binary_cross_entropy = 0.520516
Epoch 89
Loss = 3.3052e-01, PNorm = 62.1940, GNorm = 13.2991, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.515859
Epoch 90
Validation binary_cross_entropy = 0.521005
Epoch 91
Validation binary_cross_entropy = 0.613162
Epoch 92
Validation binary_cross_entropy = 0.551470
Epoch 93
Validation binary_cross_entropy = 0.508999
Epoch 94
Loss = 1.8950e-01, PNorm = 62.5298, GNorm = 8.3982, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.516890
Epoch 95
Validation binary_cross_entropy = 0.523933
Epoch 96
Validation binary_cross_entropy = 0.555466
Epoch 97
Validation binary_cross_entropy = 0.610022
Epoch 98
Validation binary_cross_entropy = 0.644820
Epoch 99
Loss = 5.0785e-01, PNorm = 62.8015, GNorm = 30.4395, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.663307
Epoch 100
Validation binary_cross_entropy = 0.594265
Epoch 101
Validation binary_cross_entropy = 0.630503
Epoch 102
Validation binary_cross_entropy = 0.682745
Epoch 103
Validation binary_cross_entropy = 0.656212
Epoch 104
Loss = 2.9644e-01, PNorm = 63.1039, GNorm = 7.7547, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.678777
Epoch 105
Validation binary_cross_entropy = 0.604060
Epoch 106
Validation binary_cross_entropy = 0.590901
Epoch 107
Validation binary_cross_entropy = 0.610189
Epoch 108
Validation binary_cross_entropy = 0.655756
Epoch 109
Loss = 2.0751e-01, PNorm = 63.3426, GNorm = 6.4845, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.737483
Epoch 110
Validation binary_cross_entropy = 0.769524
Epoch 111
Validation binary_cross_entropy = 0.766572
Epoch 112
Validation binary_cross_entropy = 0.790125
Epoch 113
Validation binary_cross_entropy = 0.798290
Epoch 114
Loss = 1.9321e-01, PNorm = 63.5871, GNorm = 8.0221, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.752086
Epoch 115
Validation binary_cross_entropy = 0.737298
Epoch 116
Validation binary_cross_entropy = 0.751972
Epoch 117
Validation binary_cross_entropy = 0.749907
Epoch 118
Validation binary_cross_entropy = 0.750613
Epoch 119
Loss = 9.3859e-02, PNorm = 63.8065, GNorm = 6.1513, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.787152
Epoch 120
Validation binary_cross_entropy = 0.933866
Epoch 121
Validation binary_cross_entropy = 0.871351
Epoch 122
Validation binary_cross_entropy = 0.890741
Epoch 123
Validation binary_cross_entropy = 0.912960
Epoch 124
Loss = 1.0629e-01, PNorm = 64.0193, GNorm = 1.6016, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.871117
Epoch 125
Validation binary_cross_entropy = 0.828675
Epoch 126
Validation binary_cross_entropy = 0.832991
Epoch 127
Validation binary_cross_entropy = 0.837478
Epoch 128
Validation binary_cross_entropy = 1.664837
Epoch 129
Loss = 2.8464e-01, PNorm = 64.2371, GNorm = 0.3719, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.817921
Epoch 130
Validation binary_cross_entropy = 0.871887
Epoch 131
Validation binary_cross_entropy = 0.779064
Epoch 132
Validation binary_cross_entropy = 0.776403
Epoch 133
Validation binary_cross_entropy = 0.811783
Epoch 134
Loss = 6.7305e-03, PNorm = 64.4516, GNorm = 0.4768, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.886831
Epoch 135
Validation binary_cross_entropy = 0.813741
Epoch 136
Validation binary_cross_entropy = 0.861652
Epoch 137
Validation binary_cross_entropy = 0.818739
Epoch 138
Validation binary_cross_entropy = 0.943717
Epoch 139
Loss = 3.5960e-01, PNorm = 64.6652, GNorm = 15.7980, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.905269
Epoch 140
Validation binary_cross_entropy = 0.912439
Epoch 141
Validation binary_cross_entropy = 0.933362
Epoch 142
Validation binary_cross_entropy = 0.897525
Epoch 143
Validation binary_cross_entropy = 1.022915
Epoch 144
Loss = 2.3658e-01, PNorm = 64.8854, GNorm = 3.2886, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.955929
Epoch 145
Validation binary_cross_entropy = 0.968319
Epoch 146
Validation binary_cross_entropy = 0.999756
Epoch 147
Validation binary_cross_entropy = 1.090187
Epoch 148
Validation binary_cross_entropy = 1.085897
Epoch 149
Loss = 5.7689e-02, PNorm = 65.0739, GNorm = 9.9875, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.123735
Epoch 150
Validation binary_cross_entropy = 1.254557
Epoch 151
Validation binary_cross_entropy = 1.370281
Epoch 152
Validation binary_cross_entropy = 0.915526
Epoch 153
Validation binary_cross_entropy = 0.836945
Epoch 154
Loss = 2.4627e-01, PNorm = 65.2506, GNorm = 25.2838, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.751850
Epoch 155
Validation binary_cross_entropy = 0.719529
Epoch 156
Validation binary_cross_entropy = 0.750609
Epoch 157
Validation binary_cross_entropy = 0.726089
Epoch 158
Validation binary_cross_entropy = 0.790785
Epoch 159
Loss = 1.4432e-01, PNorm = 65.4576, GNorm = 11.7767, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.879748
Epoch 160
Validation binary_cross_entropy = 0.837874
Epoch 161
Validation binary_cross_entropy = 0.894675
Epoch 162
Validation binary_cross_entropy = 0.941606
Epoch 163
Validation binary_cross_entropy = 0.945291
Epoch 164
Loss = 2.2761e-01, PNorm = 65.6414, GNorm = 14.2746, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.952067
Epoch 165
Validation binary_cross_entropy = 0.895626
Epoch 166
Validation binary_cross_entropy = 0.910539
Epoch 167
Validation binary_cross_entropy = 0.889922
Epoch 168
Validation binary_cross_entropy = 0.902855
Epoch 169
Loss = 1.2887e-01, PNorm = 65.7987, GNorm = 6.8735, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.940930
Epoch 170
Validation binary_cross_entropy = 0.991227
Epoch 171
Validation binary_cross_entropy = 1.022331
Epoch 172
Validation binary_cross_entropy = 1.058318
Epoch 173
Validation binary_cross_entropy = 1.121753
Epoch 174
Loss = 6.6795e-02, PNorm = 65.9719, GNorm = 5.1972, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.170942
Epoch 175
Validation binary_cross_entropy = 1.222378
Epoch 176
Validation binary_cross_entropy = 1.256991
Epoch 177
Validation binary_cross_entropy = 1.291626
Epoch 178
Validation binary_cross_entropy = 1.322537
Epoch 179
Loss = 5.1365e-02, PNorm = 66.0871, GNorm = 0.8496, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.325809
Epoch 180
Validation binary_cross_entropy = 1.282076
Epoch 181
Validation binary_cross_entropy = 1.269530
Epoch 182
Validation binary_cross_entropy = 1.308865
Epoch 183
Validation binary_cross_entropy = 1.309799
Epoch 184
Loss = 4.1625e-02, PNorm = 66.2254, GNorm = 5.9289, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.423329
Epoch 185
Validation binary_cross_entropy = 1.307485
Epoch 186
Validation binary_cross_entropy = 1.161439
Epoch 187
Validation binary_cross_entropy = 1.121874
Epoch 188
Validation binary_cross_entropy = 1.076509
Epoch 189
Loss = 2.2257e-02, PNorm = 66.4430, GNorm = 4.7852, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.118740
Epoch 190
Validation binary_cross_entropy = 1.153975
Epoch 191
Validation binary_cross_entropy = 1.135505
Epoch 192
Validation binary_cross_entropy = 1.871696
Epoch 193
Validation binary_cross_entropy = 1.197106
Epoch 194
Loss = 6.9547e-02, PNorm = 66.6345, GNorm = 4.7086, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.086398
Epoch 195
Validation binary_cross_entropy = 1.047485
Epoch 196
Validation binary_cross_entropy = 1.017334
Epoch 197
Validation binary_cross_entropy = 0.994610
Epoch 198
Validation binary_cross_entropy = 1.001133
Epoch 199
Loss = 7.5952e-02, PNorm = 66.7790, GNorm = 2.7716, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.992227
Epoch 200
Validation binary_cross_entropy = 1.010678
Epoch 201
Validation binary_cross_entropy = 1.070686
Epoch 202
Validation binary_cross_entropy = 1.129492
Epoch 203
Validation binary_cross_entropy = 1.121923
Epoch 204
Loss = 5.1196e-02, PNorm = 66.9257, GNorm = 4.5624, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.129778
Epoch 205
Validation binary_cross_entropy = 1.149121
Epoch 206
Validation binary_cross_entropy = 1.130051
Epoch 207
Validation binary_cross_entropy = 1.087840
Epoch 208
Validation binary_cross_entropy = 1.112036
Epoch 209
Loss = 6.3106e-02, PNorm = 67.0422, GNorm = 7.1700, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.169840
Epoch 210
Validation binary_cross_entropy = 1.130397
Epoch 211
Validation binary_cross_entropy = 1.167632
Epoch 212
Validation binary_cross_entropy = 1.215121
Epoch 213
Validation binary_cross_entropy = 1.275254
Epoch 214
Loss = 3.2143e-02, PNorm = 67.1440, GNorm = 2.3486, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.327437
Epoch 215
Validation binary_cross_entropy = 1.373039
Epoch 216
Validation binary_cross_entropy = 1.418504
Epoch 217
Validation binary_cross_entropy = 1.455072
Epoch 218
Validation binary_cross_entropy = 1.472501
Epoch 219
Loss = 1.5533e-02, PNorm = 67.2338, GNorm = 0.9589, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.497450
Epoch 220
Validation binary_cross_entropy = 1.494609
Epoch 221
Validation binary_cross_entropy = 1.474907
Epoch 222
Validation binary_cross_entropy = 1.472621
Epoch 223
Validation binary_cross_entropy = 1.426278
Epoch 224
Loss = 2.6566e-02, PNorm = 67.2808, GNorm = 3.7781, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.400192
Epoch 225
Validation binary_cross_entropy = 1.387362
Epoch 226
Validation binary_cross_entropy = 1.385372
Epoch 227
Validation binary_cross_entropy = 1.431732
Epoch 228
Validation binary_cross_entropy = 1.456881
Epoch 229
Loss = 3.7073e-02, PNorm = 67.3294, GNorm = 2.9982, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.437184
Epoch 230
Validation binary_cross_entropy = 1.417889
Epoch 231
Validation binary_cross_entropy = 1.410579
Epoch 232
Validation binary_cross_entropy = 1.417956
Epoch 233
Validation binary_cross_entropy = 1.408801
Epoch 234
Loss = 1.6676e-02, PNorm = 67.3824, GNorm = 3.6954, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.408130
Epoch 235
Validation binary_cross_entropy = 1.406019
Epoch 236
Validation binary_cross_entropy = 1.409197
Epoch 237
Validation binary_cross_entropy = 1.422877
Epoch 238
Validation binary_cross_entropy = 1.438347
Epoch 239
Loss = 6.9188e-03, PNorm = 67.4482, GNorm = 1.3488, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.453053
Epoch 240
Validation binary_cross_entropy = 1.459195
Epoch 241
Validation binary_cross_entropy = 1.457250
Epoch 242
Validation binary_cross_entropy = 1.461980
Epoch 243
Validation binary_cross_entropy = 1.476343
Epoch 244
Loss = 4.0599e-02, PNorm = 67.5011, GNorm = 4.1392, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.501218
Epoch 245
Validation binary_cross_entropy = 1.555287
Epoch 246
Validation binary_cross_entropy = 1.530203
Epoch 247
Validation binary_cross_entropy = 1.518142
Epoch 248
Validation binary_cross_entropy = 1.533792
Epoch 249
Loss = 6.5858e-03, PNorm = 67.5508, GNorm = 1.3141, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.554724
Epoch 250
Validation binary_cross_entropy = 1.574414
Epoch 251
Validation binary_cross_entropy = 1.587423
Epoch 252
Validation binary_cross_entropy = 1.666727
Epoch 253
Validation binary_cross_entropy = 1.622604
Epoch 254
Loss = 3.3644e-02, PNorm = 67.6083, GNorm = 1.5395, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.562349
Epoch 255
Validation binary_cross_entropy = 1.594182
Epoch 256
Validation binary_cross_entropy = 1.593796
Epoch 257
Validation binary_cross_entropy = 1.540817
Epoch 258
Validation binary_cross_entropy = 1.519085
Epoch 259
Loss = 3.9540e-02, PNorm = 67.6719, GNorm = 4.0808, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.525080
Epoch 260
Validation binary_cross_entropy = 1.538885
Epoch 261
Validation binary_cross_entropy = 1.567325
Epoch 262
Validation binary_cross_entropy = 1.572019
Epoch 263
Validation binary_cross_entropy = 1.562537
Epoch 264
Loss = 1.7724e-03, PNorm = 67.7331, GNorm = 0.1020, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 2.078972
Epoch 265
Validation binary_cross_entropy = 2.615989
Epoch 266
Validation binary_cross_entropy = 2.631007
Epoch 267
Validation binary_cross_entropy = 2.158718
Epoch 268
Validation binary_cross_entropy = 1.624089
Epoch 269
Loss = 6.6075e-02, PNorm = 67.8109, GNorm = 9.6155, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.607357
Epoch 270
Validation binary_cross_entropy = 1.586951
Epoch 271
Validation binary_cross_entropy = 1.494555
Epoch 272
Validation binary_cross_entropy = 1.502196
Epoch 273
Validation binary_cross_entropy = 2.066816
Epoch 274
Loss = 9.6837e-02, PNorm = 67.9234, GNorm = 9.1495, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 2.054718
Epoch 275
Validation binary_cross_entropy = 1.502599
Epoch 276
Validation binary_cross_entropy = 1.558738
Epoch 277
Validation binary_cross_entropy = 1.636090
Epoch 278
Validation binary_cross_entropy = 1.605986
Epoch 279
Loss = 4.8039e-02, PNorm = 68.0408, GNorm = 3.9674, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.522015
Epoch 280
Validation binary_cross_entropy = 1.467206
Epoch 281
Validation binary_cross_entropy = 1.438363
Epoch 282
Validation binary_cross_entropy = 1.424475
Epoch 283
Validation binary_cross_entropy = 1.403457
Epoch 284
Loss = 1.6941e-02, PNorm = 68.1357, GNorm = 1.0063, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.377607
Epoch 285
Validation binary_cross_entropy = 1.364464
Epoch 286
Validation binary_cross_entropy = 1.364316
Epoch 287
Validation binary_cross_entropy = 1.374057
Epoch 288
Validation binary_cross_entropy = 1.365265
Epoch 289
Loss = 1.3871e-02, PNorm = 68.1895, GNorm = 1.0043, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.372407
Epoch 290
Validation binary_cross_entropy = 1.396211
Epoch 291
Validation binary_cross_entropy = 1.420749
Epoch 292
Validation binary_cross_entropy = 1.438561
Epoch 293
Validation binary_cross_entropy = 1.448221
Epoch 294
Loss = 8.2700e-02, PNorm = 68.2436, GNorm = 5.0452, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.487032
Epoch 295
Validation binary_cross_entropy = 1.553525
Epoch 296
Validation binary_cross_entropy = 1.591794
Epoch 297
Validation binary_cross_entropy = 1.584553
Epoch 298
Validation binary_cross_entropy = 1.602410
Epoch 299
Loss = 8.0641e-04, PNorm = 68.3159, GNorm = 0.0905, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.638379
Model 0 best validation binary_cross_entropy = 0.306648 on epoch 37
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.228500
Ensemble test binary_cross_entropy = 0.228500
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.981388
Epoch 1
Validation binary_cross_entropy = 2.531147
Epoch 2
Validation binary_cross_entropy = 2.094588
Epoch 3
Validation binary_cross_entropy = 1.082965
Epoch 4
Loss = 1.5324e+00, PNorm = 57.7533, GNorm = 27.5730, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.964658
Epoch 5
Validation binary_cross_entropy = 1.164864
Epoch 6
Validation binary_cross_entropy = 0.863241
Epoch 7
Validation binary_cross_entropy = 0.640286
Epoch 8
Validation binary_cross_entropy = 0.747485
Epoch 9
Loss = 1.5139e+00, PNorm = 57.9832, GNorm = 13.2125, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.452744
Epoch 10
Validation binary_cross_entropy = 0.625520
Epoch 11
Validation binary_cross_entropy = 1.507113
Epoch 12
Validation binary_cross_entropy = 0.505974
Epoch 13
Validation binary_cross_entropy = 0.324619
Epoch 14
Loss = 1.3440e+00, PNorm = 58.2439, GNorm = 27.6335, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.335846
Epoch 15
Validation binary_cross_entropy = 0.562019
Epoch 16
Validation binary_cross_entropy = 1.131542
Epoch 17
Validation binary_cross_entropy = 1.066541
Epoch 18
Validation binary_cross_entropy = 0.742589
Epoch 19
Loss = 1.5223e+00, PNorm = 58.4580, GNorm = 19.2317, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.406924
Epoch 20
Validation binary_cross_entropy = 0.364436
Epoch 21
Validation binary_cross_entropy = 0.396520
Epoch 22
Validation binary_cross_entropy = 0.511996
Epoch 23
Validation binary_cross_entropy = 0.615547
Epoch 24
Loss = 1.2195e+00, PNorm = 58.6322, GNorm = 18.7373, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.657454
Epoch 25
Validation binary_cross_entropy = 0.656069
Epoch 26
Validation binary_cross_entropy = 0.443595
Epoch 27
Validation binary_cross_entropy = 0.303897
Epoch 28
Validation binary_cross_entropy = 0.421164
Epoch 29
Loss = 1.0122e+00, PNorm = 58.8015, GNorm = 9.7327, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.593067
Epoch 30
Validation binary_cross_entropy = 0.674354
Epoch 31
Validation binary_cross_entropy = 0.484843
Epoch 32
Validation binary_cross_entropy = 0.340561
Epoch 33
Validation binary_cross_entropy = 0.347221
Epoch 34
Loss = 7.8730e-01, PNorm = 58.9863, GNorm = 15.9322, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.480371
Epoch 35
Validation binary_cross_entropy = 0.654932
Epoch 36
Validation binary_cross_entropy = 0.644351
Epoch 37
Validation binary_cross_entropy = 0.349148
Epoch 38
Validation binary_cross_entropy = 0.306389
Epoch 39
Loss = 1.1393e+00, PNorm = 59.1732, GNorm = 15.9073, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.334391
Epoch 40
Validation binary_cross_entropy = 0.386955
Epoch 41
Validation binary_cross_entropy = 0.422242
Epoch 42
Validation binary_cross_entropy = 0.366745
Epoch 43
Validation binary_cross_entropy = 0.331147
Epoch 44
Loss = 8.3991e-01, PNorm = 59.3820, GNorm = 28.7025, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.367130
Epoch 45
Validation binary_cross_entropy = 0.444202
Epoch 46
Validation binary_cross_entropy = 0.578193
Epoch 47
Validation binary_cross_entropy = 0.438436
Epoch 48
Validation binary_cross_entropy = 0.489820
Epoch 49
Loss = 8.0411e-01, PNorm = 59.6250, GNorm = 18.5904, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.787763
Epoch 50
Validation binary_cross_entropy = 0.643282
Epoch 51
Validation binary_cross_entropy = 0.424858
Epoch 52
Validation binary_cross_entropy = 0.565418
Epoch 53
Validation binary_cross_entropy = 0.654501
Epoch 54
Loss = 8.1993e-01, PNorm = 59.9160, GNorm = 9.7877, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.368676
Epoch 55
Validation binary_cross_entropy = 0.354120
Epoch 56
Validation binary_cross_entropy = 0.362399
Epoch 57
Validation binary_cross_entropy = 0.435290
Epoch 58
Validation binary_cross_entropy = 0.368419
Epoch 59
Loss = 8.9536e-01, PNorm = 60.2449, GNorm = 16.7379, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.341164
Epoch 60
Validation binary_cross_entropy = 0.380103
Epoch 61
Validation binary_cross_entropy = 0.549599
Epoch 62
Validation binary_cross_entropy = 0.479414
Epoch 63
Validation binary_cross_entropy = 0.457788
Epoch 64
Loss = 5.7646e-01, PNorm = 60.6058, GNorm = 9.3609, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.502813
Epoch 65
Validation binary_cross_entropy = 0.459159
Epoch 66
Validation binary_cross_entropy = 0.398837
Epoch 67
Validation binary_cross_entropy = 0.392704
Epoch 68
Validation binary_cross_entropy = 0.466121
Epoch 69
Loss = 5.1359e-01, PNorm = 60.9695, GNorm = 18.6461, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.540610
Epoch 70
Validation binary_cross_entropy = 0.435412
Epoch 71
Validation binary_cross_entropy = 0.501039
Epoch 72
Validation binary_cross_entropy = 0.542005
Epoch 73
Validation binary_cross_entropy = 0.489771
Epoch 74
Loss = 3.8260e-01, PNorm = 61.3204, GNorm = 19.6067, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.918644
Epoch 75
Validation binary_cross_entropy = 0.521651
Epoch 76
Validation binary_cross_entropy = 0.518266
Epoch 77
Validation binary_cross_entropy = 0.568969
Epoch 78
Validation binary_cross_entropy = 0.530378
Epoch 79
Loss = 3.7052e-01, PNorm = 61.6939, GNorm = 19.0797, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.495313
Epoch 80
Validation binary_cross_entropy = 0.519830
Epoch 81
Validation binary_cross_entropy = 0.572233
Epoch 82
Validation binary_cross_entropy = 0.631837
Epoch 83
Validation binary_cross_entropy = 0.763602
Epoch 84
Loss = 2.1164e-01, PNorm = 62.0501, GNorm = 15.0888, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.634436
Epoch 85
Validation binary_cross_entropy = 0.609324
Epoch 86
Validation binary_cross_entropy = 0.694549
Epoch 87
Validation binary_cross_entropy = 0.643251
Epoch 88
Validation binary_cross_entropy = 0.627582
Epoch 89
Loss = 3.3011e-01, PNorm = 62.4103, GNorm = 12.3960, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.625004
Epoch 90
Validation binary_cross_entropy = 0.618616
Epoch 91
Validation binary_cross_entropy = 0.628058
Epoch 92
Validation binary_cross_entropy = 0.661835
Epoch 93
Validation binary_cross_entropy = 0.644525
Epoch 94
Loss = 4.2263e-01, PNorm = 62.7222, GNorm = 17.5644, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.710117
Epoch 95
Validation binary_cross_entropy = 0.708110
Epoch 96
Validation binary_cross_entropy = 0.756797
Epoch 97
Validation binary_cross_entropy = 0.774678
Epoch 98
Validation binary_cross_entropy = 0.745821
Epoch 99
Loss = 1.6255e-01, PNorm = 63.0356, GNorm = 3.5270, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.778494
Epoch 100
Validation binary_cross_entropy = 0.721575
Epoch 101
Validation binary_cross_entropy = 0.734817
Epoch 102
Validation binary_cross_entropy = 0.747947
Epoch 103
Validation binary_cross_entropy = 0.835184
Epoch 104
Loss = 3.5251e-01, PNorm = 63.3315, GNorm = 14.0898, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.741457
Epoch 105
Validation binary_cross_entropy = 0.789955
Epoch 106
Validation binary_cross_entropy = 0.778449
Epoch 107
Validation binary_cross_entropy = 0.805295
Epoch 108
Validation binary_cross_entropy = 0.814987
Epoch 109
Loss = 2.9448e-01, PNorm = 63.6190, GNorm = 3.2567, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.821240
Epoch 110
Validation binary_cross_entropy = 0.816546
Epoch 111
Validation binary_cross_entropy = 0.785211
Epoch 112
Validation binary_cross_entropy = 0.802088
Epoch 113
Validation binary_cross_entropy = 0.818954
Epoch 114
Loss = 1.1937e-01, PNorm = 63.8879, GNorm = 5.4108, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.912181
Epoch 115
Validation binary_cross_entropy = 0.991392
Epoch 116
Validation binary_cross_entropy = 0.889287
Epoch 117
Validation binary_cross_entropy = 0.838140
Epoch 118
Validation binary_cross_entropy = 0.743372
Epoch 119
Loss = 1.2942e-01, PNorm = 64.1298, GNorm = 3.7321, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.713571
Epoch 120
Validation binary_cross_entropy = 0.713104
Epoch 121
Validation binary_cross_entropy = 0.704673
Epoch 122
Validation binary_cross_entropy = 0.754647
Epoch 123
Validation binary_cross_entropy = 0.755794
Epoch 124
Loss = 1.1507e-01, PNorm = 64.3464, GNorm = 2.9815, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.781341
Epoch 125
Validation binary_cross_entropy = 0.810192
Epoch 126
Validation binary_cross_entropy = 1.183015
Epoch 127
Validation binary_cross_entropy = 0.902933
Epoch 128
Validation binary_cross_entropy = 1.028338
Epoch 129
Loss = 3.0807e-01, PNorm = 64.5996, GNorm = 15.4801, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.877912
Epoch 130
Validation binary_cross_entropy = 1.360523
Epoch 131
Validation binary_cross_entropy = 1.039430
Epoch 132
Validation binary_cross_entropy = 0.853846
Epoch 133
Validation binary_cross_entropy = 0.852963
Epoch 134
Loss = 1.8951e-01, PNorm = 64.8778, GNorm = 5.1232, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.896016
Epoch 135
Validation binary_cross_entropy = 0.912639
Epoch 136
Validation binary_cross_entropy = 0.882217
Epoch 137
Validation binary_cross_entropy = 0.904382
Epoch 138
Validation binary_cross_entropy = 0.919759
Epoch 139
Loss = 1.0123e-01, PNorm = 65.1107, GNorm = 10.2880, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.930065
Epoch 140
Validation binary_cross_entropy = 0.961071
Epoch 141
Validation binary_cross_entropy = 1.003065
Epoch 142
Validation binary_cross_entropy = 1.052773
Epoch 143
Validation binary_cross_entropy = 1.066207
Epoch 144
Loss = 7.2788e-02, PNorm = 65.2840, GNorm = 3.0653, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.097860
Epoch 145
Validation binary_cross_entropy = 1.066747
Epoch 146
Validation binary_cross_entropy = 1.069205
Epoch 147
Validation binary_cross_entropy = 1.101533
Epoch 148
Validation binary_cross_entropy = 1.095628
Epoch 149
Loss = 3.2196e-02, PNorm = 65.4432, GNorm = 3.5789, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.128002
Epoch 150
Validation binary_cross_entropy = 1.150851
Epoch 151
Validation binary_cross_entropy = 1.137392
Epoch 152
Validation binary_cross_entropy = 1.107576
Epoch 153
Validation binary_cross_entropy = 1.213253
Epoch 154
Loss = 6.1941e-02, PNorm = 65.5784, GNorm = 3.6217, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.157175
Epoch 155
Validation binary_cross_entropy = 1.051526
Epoch 156
Validation binary_cross_entropy = 1.026774
Epoch 157
Validation binary_cross_entropy = 1.009295
Epoch 158
Validation binary_cross_entropy = 1.025933
Epoch 159
Loss = 1.4025e-01, PNorm = 65.7233, GNorm = 4.0831, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.156215
Epoch 160
Validation binary_cross_entropy = 1.211515
Epoch 161
Validation binary_cross_entropy = 1.237648
Epoch 162
Validation binary_cross_entropy = 1.355270
Epoch 163
Validation binary_cross_entropy = 1.497897
Epoch 164
Loss = 1.1705e-01, PNorm = 65.8510, GNorm = 11.8831, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.553197
Epoch 165
Validation binary_cross_entropy = 1.591581
Epoch 166
Validation binary_cross_entropy = 2.110788
Epoch 167
Validation binary_cross_entropy = 1.552010
Epoch 168
Validation binary_cross_entropy = 1.521874
Epoch 169
Loss = 7.6815e-02, PNorm = 65.9850, GNorm = 9.4462, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.478853
Epoch 170
Validation binary_cross_entropy = 1.480126
Epoch 171
Validation binary_cross_entropy = 1.478503
Epoch 172
Validation binary_cross_entropy = 1.492253
Epoch 173
Validation binary_cross_entropy = 1.487300
Epoch 174
Loss = 2.9442e-02, PNorm = 66.1318, GNorm = 3.4518, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.512086
Epoch 175
Validation binary_cross_entropy = 1.465686
Epoch 176
Validation binary_cross_entropy = 1.281439
Epoch 177
Validation binary_cross_entropy = 1.161915
Epoch 178
Validation binary_cross_entropy = 1.103666
Epoch 179
Loss = 1.0294e-02, PNorm = 66.3106, GNorm = 0.8329, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.076189
Epoch 180
Validation binary_cross_entropy = 1.036992
Epoch 181
Validation binary_cross_entropy = 1.036807
Epoch 182
Validation binary_cross_entropy = 1.095196
Epoch 183
Validation binary_cross_entropy = 1.142469
Epoch 184
Loss = 6.1002e-02, PNorm = 66.4706, GNorm = 10.9704, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.123307
Epoch 185
Validation binary_cross_entropy = 1.094742
Epoch 186
Validation binary_cross_entropy = 1.131661
Epoch 187
Validation binary_cross_entropy = 1.248340
Epoch 188
Validation binary_cross_entropy = 1.437267
Epoch 189
Loss = 3.5639e-02, PNorm = 66.6314, GNorm = 1.1476, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.478105
Epoch 190
Validation binary_cross_entropy = 1.383216
Epoch 191
Validation binary_cross_entropy = 1.291776
Epoch 192
Validation binary_cross_entropy = 1.249802
Epoch 193
Validation binary_cross_entropy = 1.280782
Epoch 194
Loss = 2.4657e-02, PNorm = 66.7534, GNorm = 4.0088, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.310491
Epoch 195
Validation binary_cross_entropy = 1.312012
Epoch 196
Validation binary_cross_entropy = 1.373991
Epoch 197
Validation binary_cross_entropy = 1.426344
Epoch 198
Validation binary_cross_entropy = 1.318435
Epoch 199
Loss = 9.3336e-02, PNorm = 66.8452, GNorm = 11.2590, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.300304
Epoch 200
Validation binary_cross_entropy = 1.383433
Epoch 201
Validation binary_cross_entropy = 1.364503
Epoch 202
Validation binary_cross_entropy = 1.314032
Epoch 203
Validation binary_cross_entropy = 1.322879
Epoch 204
Loss = 6.1715e-02, PNorm = 66.9919, GNorm = 0.5517, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.334126
Epoch 205
Validation binary_cross_entropy = 1.368687
Epoch 206
Validation binary_cross_entropy = 1.436592
Epoch 207
Validation binary_cross_entropy = 1.444900
Epoch 208
Validation binary_cross_entropy = 1.531039
Epoch 209
Loss = 1.8646e-01, PNorm = 67.1532, GNorm = 10.2245, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.557295
Epoch 210
Validation binary_cross_entropy = 1.352538
Epoch 211
Validation binary_cross_entropy = 1.225821
Epoch 212
Validation binary_cross_entropy = 1.212188
Epoch 213
Validation binary_cross_entropy = 1.245190
Epoch 214
Loss = 6.3719e-02, PNorm = 67.3104, GNorm = 3.5961, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.292778
Epoch 215
Validation binary_cross_entropy = 1.268899
Epoch 216
Validation binary_cross_entropy = 1.319615
Epoch 217
Validation binary_cross_entropy = 1.406499
Epoch 218
Validation binary_cross_entropy = 1.460476
Epoch 219
Loss = 2.4352e-02, PNorm = 67.4663, GNorm = 0.9280, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.504312
Epoch 220
Validation binary_cross_entropy = 1.565175
Epoch 221
Validation binary_cross_entropy = 1.562093
Epoch 222
Validation binary_cross_entropy = 1.546364
Epoch 223
Validation binary_cross_entropy = 1.542759
Epoch 224
Loss = 1.2960e-02, PNorm = 67.6052, GNorm = 2.9467, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.538327
Epoch 225
Validation binary_cross_entropy = 1.500422
Epoch 226
Validation binary_cross_entropy = 1.448383
Epoch 227
Validation binary_cross_entropy = 1.480396
Epoch 228
Validation binary_cross_entropy = 1.517785
Epoch 229
Loss = 6.9675e-02, PNorm = 67.7230, GNorm = 2.6797, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.493509
Epoch 230
Validation binary_cross_entropy = 1.416076
Epoch 231
Validation binary_cross_entropy = 1.348972
Epoch 232
Validation binary_cross_entropy = 1.337253
Epoch 233
Validation binary_cross_entropy = 1.347060
Epoch 234
Loss = 1.9791e-02, PNorm = 67.8056, GNorm = 1.2702, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.334818
Epoch 235
Validation binary_cross_entropy = 1.345757
Epoch 236
Validation binary_cross_entropy = 1.359829
Epoch 237
Validation binary_cross_entropy = 1.373237
Epoch 238
Validation binary_cross_entropy = 1.394889
Epoch 239
Loss = 1.2848e-02, PNorm = 67.8791, GNorm = 2.1586, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.454294
Epoch 240
Validation binary_cross_entropy = 1.477968
Epoch 241
Validation binary_cross_entropy = 1.387851
Epoch 242
Validation binary_cross_entropy = 1.346858
Epoch 243
Validation binary_cross_entropy = 1.333694
Epoch 244
Loss = 7.7872e-02, PNorm = 67.9338, GNorm = 14.8267, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.323321
Epoch 245
Validation binary_cross_entropy = 1.423240
Epoch 246
Validation binary_cross_entropy = 1.604577
Epoch 247
Validation binary_cross_entropy = 1.676143
Epoch 248
Validation binary_cross_entropy = 1.677233
Epoch 249
Loss = 4.3139e-02, PNorm = 68.0221, GNorm = 3.6188, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.576272
Epoch 250
Validation binary_cross_entropy = 1.552648
Epoch 251
Validation binary_cross_entropy = 1.600848
Epoch 252
Validation binary_cross_entropy = 1.658512
Epoch 253
Validation binary_cross_entropy = 1.690560
Epoch 254
Loss = 2.2180e-02, PNorm = 68.1047, GNorm = 4.3357, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.724889
Epoch 255
Validation binary_cross_entropy = 1.770119
Epoch 256
Validation binary_cross_entropy = 1.817359
Epoch 257
Validation binary_cross_entropy = 1.886215
Epoch 258
Validation binary_cross_entropy = 1.820533
Epoch 259
Loss = 1.2229e-02, PNorm = 68.1620, GNorm = 1.0898, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.707502
Epoch 260
Validation binary_cross_entropy = 1.631720
Epoch 261
Validation binary_cross_entropy = 1.593170
Epoch 262
Validation binary_cross_entropy = 1.596864
Epoch 263
Validation binary_cross_entropy = 1.610882
Epoch 264
Loss = 7.1737e-03, PNorm = 68.2399, GNorm = 1.2507, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.618276
Epoch 265
Validation binary_cross_entropy = 1.603360
Epoch 266
Validation binary_cross_entropy = 1.593316
Epoch 267
Validation binary_cross_entropy = 1.650809
Epoch 268
Validation binary_cross_entropy = 1.747786
Epoch 269
Loss = 1.3292e-02, PNorm = 68.3084, GNorm = 1.3549, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.830974
Epoch 270
Validation binary_cross_entropy = 1.839016
Epoch 271
Validation binary_cross_entropy = 1.848018
Epoch 272
Validation binary_cross_entropy = 1.881690
Epoch 273
Validation binary_cross_entropy = 1.925956
Epoch 274
Loss = 5.4859e-02, PNorm = 68.3720, GNorm = 10.8503, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.977926
Epoch 275
Validation binary_cross_entropy = 2.111034
Epoch 276
Validation binary_cross_entropy = 2.145068
Epoch 277
Validation binary_cross_entropy = 1.926314
Epoch 278
Validation binary_cross_entropy = 1.775166
Epoch 279
Loss = 2.9271e-02, PNorm = 68.4475, GNorm = 4.7109, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.708982
Epoch 280
Validation binary_cross_entropy = 1.658602
Epoch 281
Validation binary_cross_entropy = 1.635298
Epoch 282
Validation binary_cross_entropy = 1.632985
Epoch 283
Validation binary_cross_entropy = 1.604409
Epoch 284
Loss = 5.9357e-02, PNorm = 68.5377, GNorm = 8.9054, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.714776
Epoch 285
Validation binary_cross_entropy = 1.758618
Epoch 286
Validation binary_cross_entropy = 2.006162
Epoch 287
Validation binary_cross_entropy = 1.750793
Epoch 288
Validation binary_cross_entropy = 1.719475
Epoch 289
Loss = 1.1177e-01, PNorm = 68.7053, GNorm = 4.8529, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.678331
Epoch 290
Validation binary_cross_entropy = 1.627874
Epoch 291
Validation binary_cross_entropy = 1.676234
Epoch 292
Validation binary_cross_entropy = 1.688622
Epoch 293
Validation binary_cross_entropy = 1.508838
Epoch 294
Loss = 4.2237e-02, PNorm = 68.8745, GNorm = 2.9525, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.418909
Epoch 295
Validation binary_cross_entropy = 1.385605
Epoch 296
Validation binary_cross_entropy = 1.356841
Epoch 297
Validation binary_cross_entropy = 1.394972
Epoch 298
Validation binary_cross_entropy = 1.472216
Epoch 299
Loss = 1.7279e-02, PNorm = 69.0188, GNorm = 1.4315, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.430523
Model 0 best validation binary_cross_entropy = 0.303897 on epoch 27
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.267992
Ensemble test binary_cross_entropy = 0.267992
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.247499
Epoch 1
Validation binary_cross_entropy = 2.250618
Epoch 2
Validation binary_cross_entropy = 1.003774
Epoch 3
Validation binary_cross_entropy = 0.951634
Epoch 4
Loss = 2.1081e+00, PNorm = 57.7613, GNorm = 20.0772, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.999161
Epoch 5
Validation binary_cross_entropy = 0.695322
Epoch 6
Validation binary_cross_entropy = 0.834009
Epoch 7
Validation binary_cross_entropy = 1.298939
Epoch 8
Validation binary_cross_entropy = 0.544832
Epoch 9
Loss = 1.4671e+00, PNorm = 57.9974, GNorm = 15.3183, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.404824
Epoch 10
Validation binary_cross_entropy = 0.626466
Epoch 11
Validation binary_cross_entropy = 1.176704
Epoch 12
Validation binary_cross_entropy = 0.970696
Epoch 13
Validation binary_cross_entropy = 0.574240
Epoch 14
Loss = 1.3299e+00, PNorm = 58.2653, GNorm = 24.1016, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.504273
Epoch 15
Validation binary_cross_entropy = 0.611609
Epoch 16
Validation binary_cross_entropy = 0.840102
Epoch 17
Validation binary_cross_entropy = 0.965944
Epoch 18
Validation binary_cross_entropy = 0.627283
Epoch 19
Loss = 1.2059e+00, PNorm = 58.4897, GNorm = 12.7251, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.379193
Epoch 20
Validation binary_cross_entropy = 0.512693
Epoch 21
Validation binary_cross_entropy = 0.744185
Epoch 22
Validation binary_cross_entropy = 0.375614
Epoch 23
Validation binary_cross_entropy = 0.316531
Epoch 24
Loss = 1.0426e+00, PNorm = 58.6593, GNorm = 18.1510, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.363169
Epoch 25
Validation binary_cross_entropy = 0.417687
Epoch 26
Validation binary_cross_entropy = 0.422922
Epoch 27
Validation binary_cross_entropy = 0.415249
Epoch 28
Validation binary_cross_entropy = 0.371506
Epoch 29
Loss = 1.0065e+00, PNorm = 58.8147, GNorm = 33.7192, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.410159
Epoch 30
Validation binary_cross_entropy = 0.800792
Epoch 31
Validation binary_cross_entropy = 0.637094
Epoch 32
Validation binary_cross_entropy = 0.410510
Epoch 33
Validation binary_cross_entropy = 0.626534
Epoch 34
Loss = 7.6595e-01, PNorm = 58.9798, GNorm = 24.1606, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.780806
Epoch 35
Validation binary_cross_entropy = 0.551379
Epoch 36
Validation binary_cross_entropy = 0.324296
Epoch 37
Validation binary_cross_entropy = 0.313538
Epoch 38
Validation binary_cross_entropy = 0.364648
Epoch 39
Loss = 8.7549e-01, PNorm = 59.1680, GNorm = 13.5426, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.540952
Epoch 40
Validation binary_cross_entropy = 0.396011
Epoch 41
Validation binary_cross_entropy = 0.332268
Epoch 42
Validation binary_cross_entropy = 0.324847
Epoch 43
Validation binary_cross_entropy = 0.505175
Epoch 44
Loss = 5.9472e-01, PNorm = 59.3859, GNorm = 24.6343, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.848321
Epoch 45
Validation binary_cross_entropy = 0.455079
Epoch 46
Validation binary_cross_entropy = 0.311122
Epoch 47
Validation binary_cross_entropy = 0.328993
Epoch 48
Validation binary_cross_entropy = 0.529009
Epoch 49
Loss = 9.2278e-01, PNorm = 59.6730, GNorm = 18.4757, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.669328
Epoch 50
Validation binary_cross_entropy = 0.524454
Epoch 51
Validation binary_cross_entropy = 0.376251
Epoch 52
Validation binary_cross_entropy = 0.553855
Epoch 53
Validation binary_cross_entropy = 0.489053
Epoch 54
Loss = 8.6868e-01, PNorm = 60.0162, GNorm = 19.1681, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.324883
Epoch 55
Validation binary_cross_entropy = 0.315158
Epoch 56
Validation binary_cross_entropy = 0.401936
Epoch 57
Validation binary_cross_entropy = 0.430628
Epoch 58
Validation binary_cross_entropy = 0.424501
Epoch 59
Loss = 6.9666e-01, PNorm = 60.3453, GNorm = 16.6520, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.402058
Epoch 60
Validation binary_cross_entropy = 0.528144
Epoch 61
Validation binary_cross_entropy = 0.459674
Epoch 62
Validation binary_cross_entropy = 0.452770
Epoch 63
Validation binary_cross_entropy = 1.194227
Epoch 64
Loss = 7.1805e-01, PNorm = 60.7445, GNorm = 21.5689, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.429589
Epoch 65
Validation binary_cross_entropy = 0.397809
Epoch 66
Validation binary_cross_entropy = 0.449537
Epoch 67
Validation binary_cross_entropy = 0.437437
Epoch 68
Validation binary_cross_entropy = 0.504200
Epoch 69
Loss = 4.4417e-01, PNorm = 61.1398, GNorm = 11.0069, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.628600
Epoch 70
Validation binary_cross_entropy = 0.546824
Epoch 71
Validation binary_cross_entropy = 0.535057
Epoch 72
Validation binary_cross_entropy = 0.825941
Epoch 73
Validation binary_cross_entropy = 0.577992
Epoch 74
Loss = 3.8192e-01, PNorm = 61.5445, GNorm = 12.8803, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.573100
Epoch 75
Validation binary_cross_entropy = 0.591049
Epoch 76
Validation binary_cross_entropy = 0.527616
Epoch 77
Validation binary_cross_entropy = 0.542123
Epoch 78
Validation binary_cross_entropy = 0.578628
Epoch 79
Loss = 5.3124e-01, PNorm = 61.9061, GNorm = 13.5822, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.615001
Epoch 80
Validation binary_cross_entropy = 0.612583
Epoch 81
Validation binary_cross_entropy = 0.701695
Epoch 82
Validation binary_cross_entropy = 0.746173
Epoch 83
Validation binary_cross_entropy = 0.645963
Epoch 84
Loss = 2.4315e-01, PNorm = 62.2923, GNorm = 22.1625, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.688833
Epoch 85
Validation binary_cross_entropy = 0.600823
Epoch 86
Validation binary_cross_entropy = 0.615570
Epoch 87
Validation binary_cross_entropy = 0.656642
Epoch 88
Validation binary_cross_entropy = 0.702842
Epoch 89
Loss = 1.9482e-01, PNorm = 62.6747, GNorm = 18.7362, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.704579
Epoch 90
Validation binary_cross_entropy = 0.728958
Epoch 91
Validation binary_cross_entropy = 0.750785
Epoch 92
Validation binary_cross_entropy = 0.688983
Epoch 93
Validation binary_cross_entropy = 0.599955
Epoch 94
Loss = 1.1569e-01, PNorm = 62.9898, GNorm = 3.4249, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.603702
Epoch 95
Validation binary_cross_entropy = 0.613138
Epoch 96
Validation binary_cross_entropy = 0.652384
Epoch 97
Validation binary_cross_entropy = 0.697488
Epoch 98
Validation binary_cross_entropy = 0.810353
Epoch 99
Loss = 1.4978e-01, PNorm = 63.2602, GNorm = 4.2283, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.887976
Epoch 100
Validation binary_cross_entropy = 0.934494
Epoch 101
Validation binary_cross_entropy = 0.978619
Epoch 102
Validation binary_cross_entropy = 0.901253
Epoch 103
Validation binary_cross_entropy = 0.818227
Epoch 104
Loss = 2.7595e-01, PNorm = 63.5403, GNorm = 11.8708, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.697714
Epoch 105
Validation binary_cross_entropy = 0.706087
Epoch 106
Validation binary_cross_entropy = 0.731995
Epoch 107
Validation binary_cross_entropy = 0.852024
Epoch 108
Validation binary_cross_entropy = 0.878252
Epoch 109
Loss = 1.1884e-01, PNorm = 63.8361, GNorm = 18.5027, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.912086
Epoch 110
Validation binary_cross_entropy = 0.988244
Epoch 111
Validation binary_cross_entropy = 0.933010
Epoch 112
Validation binary_cross_entropy = 0.872646
Epoch 113
Validation binary_cross_entropy = 0.866383
Epoch 114
Loss = 2.2874e-01, PNorm = 64.1464, GNorm = 7.5524, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.889526
Epoch 115
Validation binary_cross_entropy = 0.914126
Epoch 116
Validation binary_cross_entropy = 0.916979
Epoch 117
Validation binary_cross_entropy = 0.966991
Epoch 118
Validation binary_cross_entropy = 1.044928
Epoch 119
Loss = 5.0169e-01, PNorm = 64.4446, GNorm = 17.1298, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.095255
Epoch 120
Validation binary_cross_entropy = 1.063710
Epoch 121
Validation binary_cross_entropy = 1.128759
Epoch 122
Validation binary_cross_entropy = 1.183279
Epoch 123
Validation binary_cross_entropy = 1.247481
Epoch 124
Loss = 2.1126e-01, PNorm = 64.6932, GNorm = 1.8321, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.231982
Epoch 125
Validation binary_cross_entropy = 1.172254
Epoch 126
Validation binary_cross_entropy = 1.082110
Epoch 127
Validation binary_cross_entropy = 1.066482
Epoch 128
Validation binary_cross_entropy = 0.981260
Epoch 129
Loss = 1.8845e-01, PNorm = 64.9034, GNorm = 8.4622, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.930623
Epoch 130
Validation binary_cross_entropy = 0.881306
Epoch 131
Validation binary_cross_entropy = 0.852952
Epoch 132
Validation binary_cross_entropy = 0.863760
Epoch 133
Validation binary_cross_entropy = 0.896992
Epoch 134
Loss = 1.5441e-01, PNorm = 65.1037, GNorm = 11.2394, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.952960
Epoch 135
Validation binary_cross_entropy = 1.004144
Epoch 136
Validation binary_cross_entropy = 1.011599
Epoch 137
Validation binary_cross_entropy = 0.978171
Epoch 138
Validation binary_cross_entropy = 1.006129
Epoch 139
Loss = 9.0564e-02, PNorm = 65.3104, GNorm = 3.6017, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.072143
Epoch 140
Validation binary_cross_entropy = 1.074428
Epoch 141
Validation binary_cross_entropy = 1.030431
Epoch 142
Validation binary_cross_entropy = 1.050838
Epoch 143
Validation binary_cross_entropy = 1.216969
Epoch 144
Loss = 1.5223e-01, PNorm = 65.4804, GNorm = 4.2639, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.006059
Epoch 145
Validation binary_cross_entropy = 0.954805
Epoch 146
Validation binary_cross_entropy = 0.931475
Epoch 147
Validation binary_cross_entropy = 0.957163
Epoch 148
Validation binary_cross_entropy = 0.982201
Epoch 149
Loss = 4.5281e-02, PNorm = 65.6448, GNorm = 2.0643, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.026388
Epoch 150
Validation binary_cross_entropy = 1.071067
Epoch 151
Validation binary_cross_entropy = 1.124941
Epoch 152
Validation binary_cross_entropy = 1.155750
Epoch 153
Validation binary_cross_entropy = 1.174639
Epoch 154
Loss = 8.1907e-02, PNorm = 65.8341, GNorm = 2.4982, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.204238
Epoch 155
Validation binary_cross_entropy = 1.271105
Epoch 156
Validation binary_cross_entropy = 1.258965
Epoch 157
Validation binary_cross_entropy = 1.208795
Epoch 158
Validation binary_cross_entropy = 1.239916
Epoch 159
Loss = 3.4562e-02, PNorm = 65.9799, GNorm = 1.7539, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.274579
Epoch 160
Validation binary_cross_entropy = 1.255710
Epoch 161
Validation binary_cross_entropy = 1.279604
Epoch 162
Validation binary_cross_entropy = 1.166971
Epoch 163
Validation binary_cross_entropy = 1.091589
Epoch 164
Loss = 1.1721e-01, PNorm = 66.1053, GNorm = 2.0136, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.015025
Epoch 165
Validation binary_cross_entropy = 1.021240
Epoch 166
Validation binary_cross_entropy = 0.933157
Epoch 167
Validation binary_cross_entropy = 0.927933
Epoch 168
Validation binary_cross_entropy = 0.888611
Epoch 169
Loss = 2.3083e-01, PNorm = 66.2394, GNorm = 19.8327, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.912797
Epoch 170
Validation binary_cross_entropy = 0.964047
Epoch 171
Validation binary_cross_entropy = 1.018528
Epoch 172
Validation binary_cross_entropy = 1.093126
Epoch 173
Validation binary_cross_entropy = 1.203927
Epoch 174
Loss = 2.8026e-02, PNorm = 66.4129, GNorm = 2.3509, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.312865
Epoch 175
Validation binary_cross_entropy = 1.351412
Epoch 176
Validation binary_cross_entropy = 1.410072
Epoch 177
Validation binary_cross_entropy = 1.459654
Epoch 178
Validation binary_cross_entropy = 1.486246
Epoch 179
Loss = 1.0810e-02, PNorm = 66.5417, GNorm = 0.5500, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.517682
Epoch 180
Validation binary_cross_entropy = 1.600899
Epoch 181
Validation binary_cross_entropy = 1.788643
Epoch 182
Validation binary_cross_entropy = 1.550091
Epoch 183
Validation binary_cross_entropy = 1.512102
Epoch 184
Loss = 1.7641e-01, PNorm = 66.6558, GNorm = 4.3578, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.468871
Epoch 185
Validation binary_cross_entropy = 1.496285
Epoch 186
Validation binary_cross_entropy = 1.360027
Epoch 187
Validation binary_cross_entropy = 1.316199
Epoch 188
Validation binary_cross_entropy = 1.332741
Epoch 189
Loss = 9.7715e-02, PNorm = 66.7980, GNorm = 4.1907, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.395755
Epoch 190
Validation binary_cross_entropy = 1.393401
Epoch 191
Validation binary_cross_entropy = 1.409927
Epoch 192
Validation binary_cross_entropy = 1.376951
Epoch 193
Validation binary_cross_entropy = 1.353907
Epoch 194
Loss = 1.1433e-01, PNorm = 66.9230, GNorm = 4.6921, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.366532
Epoch 195
Validation binary_cross_entropy = 1.379800
Epoch 196
Validation binary_cross_entropy = 1.323155
Epoch 197
Validation binary_cross_entropy = 1.388796
Epoch 198
Validation binary_cross_entropy = 1.430030
Epoch 199
Loss = 2.9159e-02, PNorm = 67.0375, GNorm = 3.4621, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.417610
Epoch 200
Validation binary_cross_entropy = 1.477596
Epoch 201
Validation binary_cross_entropy = 1.493959
Epoch 202
Validation binary_cross_entropy = 1.491306
Epoch 203
Validation binary_cross_entropy = 1.509681
Epoch 204
Loss = 1.7326e-02, PNorm = 67.1544, GNorm = 0.5575, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.544450
Epoch 205
Validation binary_cross_entropy = 1.582388
Epoch 206
Validation binary_cross_entropy = 1.616603
Epoch 207
Validation binary_cross_entropy = 1.629252
Epoch 208
Validation binary_cross_entropy = 1.643984
Epoch 209
Loss = 2.0608e-02, PNorm = 67.2597, GNorm = 0.7363, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.675025
Epoch 210
Validation binary_cross_entropy = 1.714019
Epoch 211
Validation binary_cross_entropy = 1.869341
Epoch 212
Validation binary_cross_entropy = 1.869604
Epoch 213
Validation binary_cross_entropy = 1.853827
Epoch 214
Loss = 1.2349e-01, PNorm = 67.3603, GNorm = 8.7854, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.940376
Epoch 215
Validation binary_cross_entropy = 1.924721
Epoch 216
Validation binary_cross_entropy = 1.707607
Epoch 217
Validation binary_cross_entropy = 1.687168
Epoch 218
Validation binary_cross_entropy = 1.671914
Epoch 219
Loss = 2.2283e-02, PNorm = 67.4504, GNorm = 0.5120, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.708363
Epoch 220
Validation binary_cross_entropy = 1.747555
Epoch 221
Validation binary_cross_entropy = 1.708611
Epoch 222
Validation binary_cross_entropy = 1.694045
Epoch 223
Validation binary_cross_entropy = 1.676264
Epoch 224
Loss = 1.0735e-02, PNorm = 67.5246, GNorm = 1.8866, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.656949
Epoch 225
Validation binary_cross_entropy = 1.663067
Epoch 226
Validation binary_cross_entropy = 1.659949
Epoch 227
Validation binary_cross_entropy = 1.615975
Epoch 228
Validation binary_cross_entropy = 1.599656
Epoch 229
Loss = 1.8823e-02, PNorm = 67.5899, GNorm = 0.4160, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.616040
Epoch 230
Validation binary_cross_entropy = 1.664873
Epoch 231
Validation binary_cross_entropy = 1.668844
Epoch 232
Validation binary_cross_entropy = 1.683716
Epoch 233
Validation binary_cross_entropy = 1.713595
Epoch 234
Loss = 4.0213e-02, PNorm = 67.6578, GNorm = 3.5656, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.717512
Epoch 235
Validation binary_cross_entropy = 1.705163
Epoch 236
Validation binary_cross_entropy = 1.700148
Epoch 237
Validation binary_cross_entropy = 1.701795
Epoch 238
Validation binary_cross_entropy = 1.704986
Epoch 239
Loss = 2.7532e-02, PNorm = 67.7167, GNorm = 1.0524, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.733067
Epoch 240
Validation binary_cross_entropy = 1.736531
Epoch 241
Validation binary_cross_entropy = 1.711684
Epoch 242
Validation binary_cross_entropy = 1.708199
Epoch 243
Validation binary_cross_entropy = 1.709622
Epoch 244
Loss = 3.6476e-03, PNorm = 67.7631, GNorm = 0.7510, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.718326
Epoch 245
Validation binary_cross_entropy = 1.726207
Epoch 246
Validation binary_cross_entropy = 1.715223
Epoch 247
Validation binary_cross_entropy = 1.697866
Epoch 248
Validation binary_cross_entropy = 1.687109
Epoch 249
Loss = 1.5105e-02, PNorm = 67.7878, GNorm = 0.1552, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.694947
Epoch 250
Validation binary_cross_entropy = 1.725573
Epoch 251
Validation binary_cross_entropy = 1.705145
Epoch 252
Validation binary_cross_entropy = 1.660370
Epoch 253
Validation binary_cross_entropy = 1.621782
Epoch 254
Loss = 1.3783e-02, PNorm = 67.8226, GNorm = 1.2465, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.587237
Epoch 255
Validation binary_cross_entropy = 1.651287
Epoch 256
Validation binary_cross_entropy = 1.754622
Epoch 257
Validation binary_cross_entropy = 1.896175
Epoch 258
Validation binary_cross_entropy = 1.906684
Epoch 259
Loss = 2.8965e-02, PNorm = 67.8901, GNorm = 0.7545, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.898538
Epoch 260
Validation binary_cross_entropy = 1.950930
Epoch 261
Validation binary_cross_entropy = 1.990693
Epoch 262
Validation binary_cross_entropy = 2.014199
Epoch 263
Validation binary_cross_entropy = 2.023509
Epoch 264
Loss = 2.3064e-02, PNorm = 67.9349, GNorm = 1.5750, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 2.026098
Epoch 265
Validation binary_cross_entropy = 2.031047
Epoch 266
Validation binary_cross_entropy = 2.022887
Epoch 267
Validation binary_cross_entropy = 2.042295
Epoch 268
Validation binary_cross_entropy = 2.062797
Epoch 269
Loss = 1.7936e-03, PNorm = 67.9787, GNorm = 0.3118, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 2.077718
Epoch 270
Validation binary_cross_entropy = 2.076493
Epoch 271
Validation binary_cross_entropy = 2.070811
Epoch 272
Validation binary_cross_entropy = 2.023286
Epoch 273
Validation binary_cross_entropy = 1.944129
Epoch 274
Loss = 1.0368e-02, PNorm = 68.0341, GNorm = 3.8752, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.867422
Epoch 275
Validation binary_cross_entropy = 1.823863
Epoch 276
Validation binary_cross_entropy = 1.819942
Epoch 277
Validation binary_cross_entropy = 1.839972
Epoch 278
Validation binary_cross_entropy = 1.938732
Epoch 279
Loss = 3.5330e-02, PNorm = 68.0860, GNorm = 4.6646, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.991625
Epoch 280
Validation binary_cross_entropy = 1.964189
Epoch 281
Validation binary_cross_entropy = 1.910825
Epoch 282
Validation binary_cross_entropy = 1.881287
Epoch 283
Validation binary_cross_entropy = 1.877146
Epoch 284
Loss = 1.0534e-02, PNorm = 68.1251, GNorm = 0.9106, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.902506
Epoch 285
Validation binary_cross_entropy = 1.930740
Epoch 286
Validation binary_cross_entropy = 1.962291
Epoch 287
Validation binary_cross_entropy = 2.017957
Epoch 288
Validation binary_cross_entropy = 2.070786
Epoch 289
Loss = 9.2522e-03, PNorm = 68.1769, GNorm = 1.3699, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 2.081011
Epoch 290
Validation binary_cross_entropy = 2.030008
Epoch 291
Validation binary_cross_entropy = 2.010279
Epoch 292
Validation binary_cross_entropy = 2.002204
Epoch 293
Validation binary_cross_entropy = 1.988486
Epoch 294
Loss = 1.3387e-02, PNorm = 68.2122, GNorm = 1.2382, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.964923
Epoch 295
Validation binary_cross_entropy = 1.961423
Epoch 296
Validation binary_cross_entropy = 2.007063
Epoch 297
Validation binary_cross_entropy = 2.081482
Epoch 298
Validation binary_cross_entropy = 2.141458
Epoch 299
Loss = 1.4794e-02, PNorm = 68.2588, GNorm = 5.1901, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 2.152826
Model 0 best validation binary_cross_entropy = 0.311122 on epoch 46
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.265076
Ensemble test binary_cross_entropy = 0.265076
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.372274
Epoch 1
Validation binary_cross_entropy = 1.765171
Epoch 2
Validation binary_cross_entropy = 1.114738
Epoch 3
Validation binary_cross_entropy = 0.917714
Epoch 4
Loss = 1.9961e+00, PNorm = 57.7544, GNorm = 40.1416, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.846285
Epoch 5
Validation binary_cross_entropy = 1.397307
Epoch 6
Validation binary_cross_entropy = 1.033301
Epoch 7
Validation binary_cross_entropy = 0.445860
Epoch 8
Validation binary_cross_entropy = 0.899891
Epoch 9
Loss = 1.3386e+00, PNorm = 57.9572, GNorm = 18.4077, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.710279
Epoch 10
Validation binary_cross_entropy = 0.495473
Epoch 11
Validation binary_cross_entropy = 0.408580
Epoch 12
Validation binary_cross_entropy = 0.416017
Epoch 13
Validation binary_cross_entropy = 0.598716
Epoch 14
Loss = 1.4458e+00, PNorm = 58.1919, GNorm = 15.0382, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.820461
Epoch 15
Validation binary_cross_entropy = 0.618331
Epoch 16
Validation binary_cross_entropy = 0.442638
Epoch 17
Validation binary_cross_entropy = 0.341110
Epoch 18
Validation binary_cross_entropy = 0.480949
Epoch 19
Loss = 1.0625e+00, PNorm = 58.3889, GNorm = 23.2116, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.772866
Epoch 20
Validation binary_cross_entropy = 0.662240
Epoch 21
Validation binary_cross_entropy = 0.499428
Epoch 22
Validation binary_cross_entropy = 0.371135
Epoch 23
Validation binary_cross_entropy = 0.358446
Epoch 24
Loss = 1.0125e+00, PNorm = 58.5580, GNorm = 23.8897, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.511603
Epoch 25
Validation binary_cross_entropy = 0.683581
Epoch 26
Validation binary_cross_entropy = 0.517302
Epoch 27
Validation binary_cross_entropy = 0.285117
Epoch 28
Validation binary_cross_entropy = 0.280950
Epoch 29
Loss = 1.0430e+00, PNorm = 58.7377, GNorm = 24.0755, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.376179
Epoch 30
Validation binary_cross_entropy = 0.680654
Epoch 31
Validation binary_cross_entropy = 0.644323
Epoch 32
Validation binary_cross_entropy = 0.414335
Epoch 33
Validation binary_cross_entropy = 0.349702
Epoch 34
Loss = 5.9224e-01, PNorm = 58.9268, GNorm = 17.0140, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.381619
Epoch 35
Validation binary_cross_entropy = 0.416871
Epoch 36
Validation binary_cross_entropy = 0.414013
Epoch 37
Validation binary_cross_entropy = 0.404263
Epoch 38
Validation binary_cross_entropy = 0.414159
Epoch 39
Loss = 9.4056e-01, PNorm = 59.1328, GNorm = 10.3990, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.467645
Epoch 40
Validation binary_cross_entropy = 0.537232
Epoch 41
Validation binary_cross_entropy = 0.372857
Epoch 42
Validation binary_cross_entropy = 0.312931
Epoch 43
Validation binary_cross_entropy = 0.433420
Epoch 44
Loss = 9.9317e-01, PNorm = 59.3342, GNorm = 16.7296, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.582061
Epoch 45
Validation binary_cross_entropy = 0.533957
Epoch 46
Validation binary_cross_entropy = 0.371689
Epoch 47
Validation binary_cross_entropy = 0.318837
Epoch 48
Validation binary_cross_entropy = 0.339859
Epoch 49
Loss = 4.3459e-01, PNorm = 59.5621, GNorm = 10.0157, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.534030
Epoch 50
Validation binary_cross_entropy = 0.580967
Epoch 51
Validation binary_cross_entropy = 0.403586
Epoch 52
Validation binary_cross_entropy = 0.367188
Epoch 53
Validation binary_cross_entropy = 0.407581
Epoch 54
Loss = 5.9005e-01, PNorm = 59.8072, GNorm = 20.3735, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.606861
Epoch 55
Validation binary_cross_entropy = 0.386847
Epoch 56
Validation binary_cross_entropy = 0.297630
Epoch 57
Validation binary_cross_entropy = 0.300974
Epoch 58
Validation binary_cross_entropy = 0.325713
Epoch 59
Loss = 5.3519e-01, PNorm = 60.0963, GNorm = 14.0872, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.376909
Epoch 60
Validation binary_cross_entropy = 0.524929
Epoch 61
Validation binary_cross_entropy = 0.602222
Epoch 62
Validation binary_cross_entropy = 0.504331
Epoch 63
Validation binary_cross_entropy = 0.567994
Epoch 64
Loss = 6.0618e-01, PNorm = 60.4210, GNorm = 15.7379, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.694249
Epoch 65
Validation binary_cross_entropy = 0.376067
Epoch 66
Validation binary_cross_entropy = 0.325765
Epoch 67
Validation binary_cross_entropy = 0.334063
Epoch 68
Validation binary_cross_entropy = 0.440925
Epoch 69
Loss = 4.5132e-01, PNorm = 60.7570, GNorm = 14.5783, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.569735
Epoch 70
Validation binary_cross_entropy = 0.554595
Epoch 71
Validation binary_cross_entropy = 0.426810
Epoch 72
Validation binary_cross_entropy = 0.424266
Epoch 73
Validation binary_cross_entropy = 0.427235
Epoch 74
Loss = 5.0813e-01, PNorm = 61.1145, GNorm = 6.9753, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.688342
Epoch 75
Validation binary_cross_entropy = 0.459732
Epoch 76
Validation binary_cross_entropy = 0.368740
Epoch 77
Validation binary_cross_entropy = 0.370485
Epoch 78
Validation binary_cross_entropy = 0.374123
Epoch 79
Loss = 4.5453e-01, PNorm = 61.4663, GNorm = 8.9887, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.456005
Epoch 80
Validation binary_cross_entropy = 0.460064
Epoch 81
Validation binary_cross_entropy = 0.428617
Epoch 82
Validation binary_cross_entropy = 0.442466
Epoch 83
Validation binary_cross_entropy = 0.485178
Epoch 84
Loss = 3.1428e-01, PNorm = 61.8566, GNorm = 12.3547, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.463203
Epoch 85
Validation binary_cross_entropy = 0.471689
Epoch 86
Validation binary_cross_entropy = 0.476624
Epoch 87
Validation binary_cross_entropy = 0.498458
Epoch 88
Validation binary_cross_entropy = 0.604419
Epoch 89
Loss = 4.1198e-01, PNorm = 62.2218, GNorm = 8.3957, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.602897
Epoch 90
Validation binary_cross_entropy = 0.573615
Epoch 91
Validation binary_cross_entropy = 0.532261
Epoch 92
Validation binary_cross_entropy = 0.550233
Epoch 93
Validation binary_cross_entropy = 0.525250
Epoch 94
Loss = 5.0200e-01, PNorm = 62.5762, GNorm = 36.8103, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.520970
Epoch 95
Validation binary_cross_entropy = 0.531698
Epoch 96
Validation binary_cross_entropy = 0.614377
Epoch 97
Validation binary_cross_entropy = 0.639146
Epoch 98
Validation binary_cross_entropy = 0.564853
Epoch 99
Loss = 2.7878e-01, PNorm = 62.9444, GNorm = 18.9047, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.536136
Epoch 100
Validation binary_cross_entropy = 0.522800
Epoch 101
Validation binary_cross_entropy = 0.582492
Epoch 102
Validation binary_cross_entropy = 0.522230
Epoch 103
Validation binary_cross_entropy = 0.592629
Epoch 104
Loss = 4.8957e-01, PNorm = 63.2983, GNorm = 31.4177, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.606170
Epoch 105
Validation binary_cross_entropy = 0.726777
Epoch 106
Validation binary_cross_entropy = 0.647310
Epoch 107
Validation binary_cross_entropy = 0.636345
Epoch 108
Validation binary_cross_entropy = 0.663990
Epoch 109
Loss = 2.4559e-01, PNorm = 63.6849, GNorm = 11.8249, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.586164
Epoch 110
Validation binary_cross_entropy = 0.559943
Epoch 111
Validation binary_cross_entropy = 0.540313
Epoch 112
Validation binary_cross_entropy = 0.565775
Epoch 113
Validation binary_cross_entropy = 0.550424
Epoch 114
Loss = 3.1865e-01, PNorm = 64.0078, GNorm = 24.2787, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.543845
Epoch 115
Validation binary_cross_entropy = 0.528600
Epoch 116
Validation binary_cross_entropy = 0.533175
Epoch 117
Validation binary_cross_entropy = 0.553915
Epoch 118
Validation binary_cross_entropy = 0.570545
Epoch 119
Loss = 9.2772e-02, PNorm = 64.2899, GNorm = 5.7969, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.601847
Epoch 120
Validation binary_cross_entropy = 0.666847
Epoch 121
Validation binary_cross_entropy = 0.765831
Epoch 122
Validation binary_cross_entropy = 0.781417
Epoch 123
Validation binary_cross_entropy = 0.732164
Epoch 124
Loss = 1.1981e-01, PNorm = 64.5153, GNorm = 8.3487, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.649515
Epoch 125
Validation binary_cross_entropy = 0.609181
Epoch 126
Validation binary_cross_entropy = 0.574854
Epoch 127
Validation binary_cross_entropy = 0.563707
Epoch 128
Validation binary_cross_entropy = 0.589100
Epoch 129
Loss = 1.6527e-01, PNorm = 64.7450, GNorm = 18.2307, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.594794
Epoch 130
Validation binary_cross_entropy = 0.608140
Epoch 131
Validation binary_cross_entropy = 0.638709
Epoch 132
Validation binary_cross_entropy = 0.667404
Epoch 133
Validation binary_cross_entropy = 0.728377
Epoch 134
Loss = 1.7140e-01, PNorm = 64.9971, GNorm = 5.3572, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.746495
Epoch 135
Validation binary_cross_entropy = 0.798777
Epoch 136
Validation binary_cross_entropy = 0.776835
Epoch 137
Validation binary_cross_entropy = 0.764121
Epoch 138
Validation binary_cross_entropy = 0.749123
Epoch 139
Loss = 1.0066e-01, PNorm = 65.2306, GNorm = 4.5143, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.670371
Epoch 140
Validation binary_cross_entropy = 0.648220
Epoch 141
Validation binary_cross_entropy = 0.736655
Epoch 142
Validation binary_cross_entropy = 0.750653
Epoch 143
Validation binary_cross_entropy = 0.670608
Epoch 144
Loss = 8.5790e-02, PNorm = 65.4602, GNorm = 6.2681, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.703209
Epoch 145
Validation binary_cross_entropy = 0.725654
Epoch 146
Validation binary_cross_entropy = 0.800767
Epoch 147
Validation binary_cross_entropy = 0.844752
Epoch 148
Validation binary_cross_entropy = 0.792329
Epoch 149
Loss = 1.3569e-01, PNorm = 65.7154, GNorm = 11.6684, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.754368
Epoch 150
Validation binary_cross_entropy = 0.717958
Epoch 151
Validation binary_cross_entropy = 0.692877
Epoch 152
Validation binary_cross_entropy = 0.755208
Epoch 153
Validation binary_cross_entropy = 0.739858
Epoch 154
Loss = 5.3056e-02, PNorm = 65.9490, GNorm = 4.2979, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.753843
Epoch 155
Validation binary_cross_entropy = 0.783613
Epoch 156
Validation binary_cross_entropy = 0.868306
Epoch 157
Validation binary_cross_entropy = 0.837171
Epoch 158
Validation binary_cross_entropy = 0.773504
Epoch 159
Loss = 2.5706e-02, PNorm = 66.1531, GNorm = 3.0380, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.736322
Epoch 160
Validation binary_cross_entropy = 0.752633
Epoch 161
Validation binary_cross_entropy = 0.790923
Epoch 162
Validation binary_cross_entropy = 0.818967
Epoch 163
Validation binary_cross_entropy = 0.809705
Epoch 164
Loss = 1.3030e-01, PNorm = 66.3925, GNorm = 2.6738, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.811446
Epoch 165
Validation binary_cross_entropy = 0.827995
Epoch 166
Validation binary_cross_entropy = 0.856684
Epoch 167
Validation binary_cross_entropy = 0.888881
Epoch 168
Validation binary_cross_entropy = 0.897199
Epoch 169
Loss = 1.2831e-01, PNorm = 66.5826, GNorm = 3.7922, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.928571
Epoch 170
Validation binary_cross_entropy = 0.993439
Epoch 171
Validation binary_cross_entropy = 1.064574
Epoch 172
Validation binary_cross_entropy = 1.107019
Epoch 173
Validation binary_cross_entropy = 0.958717
Epoch 174
Loss = 2.6881e-02, PNorm = 66.7331, GNorm = 1.6782, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.906830
Epoch 175
Validation binary_cross_entropy = 0.851030
Epoch 176
Validation binary_cross_entropy = 0.857020
Epoch 177
Validation binary_cross_entropy = 0.837349
Epoch 178
Validation binary_cross_entropy = 0.822244
Epoch 179
Loss = 1.5853e-02, PNorm = 66.8736, GNorm = 1.6710, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.844590
Epoch 180
Validation binary_cross_entropy = 0.876322
Epoch 181
Validation binary_cross_entropy = 0.917064
Epoch 182
Validation binary_cross_entropy = 0.956935
Epoch 183
Validation binary_cross_entropy = 1.020972
Epoch 184
Loss = 8.9358e-02, PNorm = 67.0621, GNorm = 6.2970, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.039627
Epoch 185
Validation binary_cross_entropy = 0.968840
Epoch 186
Validation binary_cross_entropy = 0.943022
Epoch 187
Validation binary_cross_entropy = 1.021707
Epoch 188
Validation binary_cross_entropy = 1.091128
Epoch 189
Loss = 3.0437e-02, PNorm = 67.2358, GNorm = 1.3213, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.048303
Epoch 190
Validation binary_cross_entropy = 1.033459
Epoch 191
Validation binary_cross_entropy = 1.053571
Epoch 192
Validation binary_cross_entropy = 1.068142
Epoch 193
Validation binary_cross_entropy = 1.054192
Epoch 194
Loss = 3.5858e-02, PNorm = 67.3954, GNorm = 4.7741, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.011598
Epoch 195
Validation binary_cross_entropy = 1.014903
Epoch 196
Validation binary_cross_entropy = 1.083654
Epoch 197
Validation binary_cross_entropy = 1.063218
Epoch 198
Validation binary_cross_entropy = 1.052144
Epoch 199
Loss = 4.2646e-02, PNorm = 67.5510, GNorm = 3.9933, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.105542
Epoch 200
Validation binary_cross_entropy = 1.191490
Epoch 201
Validation binary_cross_entropy = 1.102427
Epoch 202
Validation binary_cross_entropy = 1.152099
Epoch 203
Validation binary_cross_entropy = 1.178393
Epoch 204
Loss = 3.7329e-02, PNorm = 67.6864, GNorm = 2.1488, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.193894
Epoch 205
Validation binary_cross_entropy = 1.166479
Epoch 206
Validation binary_cross_entropy = 1.154717
Epoch 207
Validation binary_cross_entropy = 1.173139
Epoch 208
Validation binary_cross_entropy = 1.224101
Epoch 209
Loss = 1.0913e-02, PNorm = 67.8052, GNorm = 0.9974, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.249257
Epoch 210
Validation binary_cross_entropy = 1.239214
Epoch 211
Validation binary_cross_entropy = 1.228933
Epoch 212
Validation binary_cross_entropy = 1.349264
Epoch 213
Validation binary_cross_entropy = 1.240037
Epoch 214
Loss = 5.5150e-02, PNorm = 67.9093, GNorm = 6.8243, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.102957
Epoch 215
Validation binary_cross_entropy = 1.101706
Epoch 216
Validation binary_cross_entropy = 1.065508
Epoch 217
Validation binary_cross_entropy = 1.053639
Epoch 218
Validation binary_cross_entropy = 1.092963
Epoch 219
Loss = 4.8803e-02, PNorm = 68.0418, GNorm = 10.1633, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.140100
Epoch 220
Validation binary_cross_entropy = 1.152680
Epoch 221
Validation binary_cross_entropy = 1.110152
Epoch 222
Validation binary_cross_entropy = 1.083804
Epoch 223
Validation binary_cross_entropy = 1.097228
Epoch 224
Loss = 2.7753e-02, PNorm = 68.1593, GNorm = 0.6492, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.135673
Epoch 225
Validation binary_cross_entropy = 1.196235
Epoch 226
Validation binary_cross_entropy = 1.219306
Epoch 227
Validation binary_cross_entropy = 1.205275
Epoch 228
Validation binary_cross_entropy = 1.201239
Epoch 229
Loss = 2.6483e-02, PNorm = 68.2401, GNorm = 2.3848, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.226372
Epoch 230
Validation binary_cross_entropy = 1.272670
Epoch 231
Validation binary_cross_entropy = 1.378678
Epoch 232
Validation binary_cross_entropy = 1.476023
Epoch 233
Validation binary_cross_entropy = 1.437916
Epoch 234
Loss = 6.7183e-03, PNorm = 68.3139, GNorm = 1.3408, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.388083
Epoch 235
Validation binary_cross_entropy = 1.359808
Epoch 236
Validation binary_cross_entropy = 1.359204
Epoch 237
Validation binary_cross_entropy = 1.391510
Epoch 238
Validation binary_cross_entropy = 1.405463
Epoch 239
Loss = 1.1345e-02, PNorm = 68.3675, GNorm = 0.5437, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.423407
Epoch 240
Validation binary_cross_entropy = 1.423344
Epoch 241
Validation binary_cross_entropy = 1.451750
Epoch 242
Validation binary_cross_entropy = 1.452051
Epoch 243
Validation binary_cross_entropy = 1.401646
Epoch 244
Loss = 1.3160e-02, PNorm = 68.4125, GNorm = 2.1836, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.395756
Epoch 245
Validation binary_cross_entropy = 1.404176
Epoch 246
Validation binary_cross_entropy = 1.447727
Epoch 247
Validation binary_cross_entropy = 1.516730
Epoch 248
Validation binary_cross_entropy = 1.493657
Epoch 249
Loss = 1.6698e-02, PNorm = 68.4694, GNorm = 2.1404, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.428123
Epoch 250
Validation binary_cross_entropy = 1.400063
Epoch 251
Validation binary_cross_entropy = 1.416940
Epoch 252
Validation binary_cross_entropy = 1.459119
Epoch 253
Validation binary_cross_entropy = 1.481124
Epoch 254
Loss = 3.0880e-03, PNorm = 68.5288, GNorm = 0.5414, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.504292
Epoch 255
Validation binary_cross_entropy = 1.548302
Epoch 256
Validation binary_cross_entropy = 1.593508
Epoch 257
Validation binary_cross_entropy = 1.626361
Epoch 258
Validation binary_cross_entropy = 1.636689
Epoch 259
Loss = 9.9418e-03, PNorm = 68.5920, GNorm = 2.9594, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.617899
Epoch 260
Validation binary_cross_entropy = 1.533990
Epoch 261
Validation binary_cross_entropy = 1.477795
Epoch 262
Validation binary_cross_entropy = 1.504706
Epoch 263
Validation binary_cross_entropy = 1.320458
Epoch 264
Loss = 7.7075e-02, PNorm = 68.6820, GNorm = 0.4250, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.341570
Epoch 265
Validation binary_cross_entropy = 1.352645
Epoch 266
Validation binary_cross_entropy = 1.128270
Epoch 267
Validation binary_cross_entropy = 1.104814
Epoch 268
Validation binary_cross_entropy = 1.127592
Epoch 269
Loss = 1.0201e-01, PNorm = 68.8089, GNorm = 6.9471, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.145649
Epoch 270
Validation binary_cross_entropy = 1.072486
Epoch 271
Validation binary_cross_entropy = 1.034263
Epoch 272
Validation binary_cross_entropy = 1.019293
Epoch 273
Validation binary_cross_entropy = 1.058202
Epoch 274
Loss = 1.1126e-01, PNorm = 68.9458, GNorm = 7.9780, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.082799
Epoch 275
Validation binary_cross_entropy = 1.041695
Epoch 276
Validation binary_cross_entropy = 1.033726
Epoch 277
Validation binary_cross_entropy = 1.114161
Epoch 278
Validation binary_cross_entropy = 1.198188
Epoch 279
Loss = 8.5061e-02, PNorm = 69.0726, GNorm = 7.3832, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.137490
Epoch 280
Validation binary_cross_entropy = 1.360253
Epoch 281
Validation binary_cross_entropy = 1.318433
Epoch 282
Validation binary_cross_entropy = 1.235193
Epoch 283
Validation binary_cross_entropy = 1.437996
Epoch 284
Loss = 3.0475e-01, PNorm = 69.2708, GNorm = 15.1243, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.342338
Epoch 285
Validation binary_cross_entropy = 1.389559
Epoch 286
Validation binary_cross_entropy = 1.437542
Epoch 287
Validation binary_cross_entropy = 1.297091
Epoch 288
Validation binary_cross_entropy = 1.232595
Epoch 289
Loss = 3.2417e-02, PNorm = 69.4938, GNorm = 1.3281, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.289762
Epoch 290
Validation binary_cross_entropy = 1.282273
Epoch 291
Validation binary_cross_entropy = 1.255383
Epoch 292
Validation binary_cross_entropy = 1.369989
Epoch 293
Validation binary_cross_entropy = 1.435600
Epoch 294
Loss = 1.3895e-01, PNorm = 69.6938, GNorm = 6.0164, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.338514
Epoch 295
Validation binary_cross_entropy = 1.152408
Epoch 296
Validation binary_cross_entropy = 1.090293
Epoch 297
Validation binary_cross_entropy = 1.065601
Epoch 298
Validation binary_cross_entropy = 1.060485
Epoch 299
Loss = 1.8925e-02, PNorm = 69.8072, GNorm = 0.7240, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.075957
Model 0 best validation binary_cross_entropy = 0.280950 on epoch 28
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.217307
Ensemble test binary_cross_entropy = 0.217307
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 3.027648
Epoch 1
Validation binary_cross_entropy = 1.902278
Epoch 2
Validation binary_cross_entropy = 1.090730
Epoch 3
Validation binary_cross_entropy = 1.513964
Epoch 4
Loss = 1.5430e+00, PNorm = 57.7639, GNorm = 22.4858, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 1.168036
Epoch 5
Validation binary_cross_entropy = 0.715005
Epoch 6
Validation binary_cross_entropy = 1.158316
Epoch 7
Validation binary_cross_entropy = 0.623371
Epoch 8
Validation binary_cross_entropy = 0.655363
Epoch 9
Loss = 1.2152e+00, PNorm = 57.9864, GNorm = 28.2647, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 1.377567
Epoch 10
Validation binary_cross_entropy = 0.665332
Epoch 11
Validation binary_cross_entropy = 0.411032
Epoch 12
Validation binary_cross_entropy = 0.450884
Epoch 13
Validation binary_cross_entropy = 0.704075
Epoch 14
Loss = 1.2852e+00, PNorm = 58.2435, GNorm = 28.9742, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.746753
Epoch 15
Validation binary_cross_entropy = 0.356344
Epoch 16
Validation binary_cross_entropy = 0.397047
Epoch 17
Validation binary_cross_entropy = 0.564723
Epoch 18
Validation binary_cross_entropy = 0.504079
Epoch 19
Loss = 1.0855e+00, PNorm = 58.4497, GNorm = 12.3253, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.334632
Epoch 20
Validation binary_cross_entropy = 0.417922
Epoch 21
Validation binary_cross_entropy = 0.612005
Epoch 22
Validation binary_cross_entropy = 0.558304
Epoch 23
Validation binary_cross_entropy = 0.441349
Epoch 24
Loss = 1.0332e+00, PNorm = 58.6257, GNorm = 27.3083, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.373145
Epoch 25
Validation binary_cross_entropy = 0.621104
Epoch 26
Validation binary_cross_entropy = 0.518611
Epoch 27
Validation binary_cross_entropy = 0.308898
Epoch 28
Validation binary_cross_entropy = 0.607896
Epoch 29
Loss = 1.0758e+00, PNorm = 58.8218, GNorm = 18.7050, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.847435
Epoch 30
Validation binary_cross_entropy = 0.438128
Epoch 31
Validation binary_cross_entropy = 0.341436
Epoch 32
Validation binary_cross_entropy = 0.490780
Epoch 33
Validation binary_cross_entropy = 0.511527
Epoch 34
Loss = 7.2848e-01, PNorm = 59.0222, GNorm = 19.9490, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.380921
Epoch 35
Validation binary_cross_entropy = 0.380784
Epoch 36
Validation binary_cross_entropy = 0.490169
Epoch 37
Validation binary_cross_entropy = 0.431796
Epoch 38
Validation binary_cross_entropy = 0.335859
Epoch 39
Loss = 1.0506e+00, PNorm = 59.2385, GNorm = 15.8692, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.318983
Epoch 40
Validation binary_cross_entropy = 0.430020
Epoch 41
Validation binary_cross_entropy = 0.535102
Epoch 42
Validation binary_cross_entropy = 0.409625
Epoch 43
Validation binary_cross_entropy = 0.395195
Epoch 44
Loss = 5.9226e-01, PNorm = 59.4547, GNorm = 9.3952, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.643652
Epoch 45
Validation binary_cross_entropy = 0.403867
Epoch 46
Validation binary_cross_entropy = 0.335048
Epoch 47
Validation binary_cross_entropy = 0.409612
Epoch 48
Validation binary_cross_entropy = 0.618082
Epoch 49
Loss = 6.6978e-01, PNorm = 59.7343, GNorm = 15.5021, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.396040
Epoch 50
Validation binary_cross_entropy = 0.441124
Epoch 51
Validation binary_cross_entropy = 0.513768
Epoch 52
Validation binary_cross_entropy = 0.307616
Epoch 53
Validation binary_cross_entropy = 0.301799
Epoch 54
Loss = 7.0891e-01, PNorm = 60.0293, GNorm = 10.5644, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.457081
Epoch 55
Validation binary_cross_entropy = 0.632675
Epoch 56
Validation binary_cross_entropy = 0.365670
Epoch 57
Validation binary_cross_entropy = 0.312732
Epoch 58
Validation binary_cross_entropy = 0.354951
Epoch 59
Loss = 6.3935e-01, PNorm = 60.3524, GNorm = 19.1007, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.526660
Epoch 60
Validation binary_cross_entropy = 0.377800
Epoch 61
Validation binary_cross_entropy = 0.344736
Epoch 62
Validation binary_cross_entropy = 0.484342
Epoch 63
Validation binary_cross_entropy = 0.553905
Epoch 64
Loss = 5.7934e-01, PNorm = 60.6966, GNorm = 11.9669, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.382982
Epoch 65
Validation binary_cross_entropy = 0.405613
Epoch 66
Validation binary_cross_entropy = 0.733186
Epoch 67
Validation binary_cross_entropy = 0.448252
Epoch 68
Validation binary_cross_entropy = 0.404088
Epoch 69
Loss = 5.1779e-01, PNorm = 61.0620, GNorm = 11.5372, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.458812
Epoch 70
Validation binary_cross_entropy = 0.440023
Epoch 71
Validation binary_cross_entropy = 0.415634
Epoch 72
Validation binary_cross_entropy = 0.467465
Epoch 73
Validation binary_cross_entropy = 0.562407
Epoch 74
Loss = 1.9472e-01, PNorm = 61.4434, GNorm = 2.4205, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.497524
Epoch 75
Validation binary_cross_entropy = 0.526222
Epoch 76
Validation binary_cross_entropy = 0.681435
Epoch 77
Validation binary_cross_entropy = 0.808656
Epoch 78
Validation binary_cross_entropy = 0.557101
Epoch 79
Loss = 3.9956e-01, PNorm = 61.8275, GNorm = 10.7634, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.634936
Epoch 80
Validation binary_cross_entropy = 0.567753
Epoch 81
Validation binary_cross_entropy = 0.521540
Epoch 82
Validation binary_cross_entropy = 0.551690
Epoch 83
Validation binary_cross_entropy = 0.723434
Epoch 84
Loss = 3.2675e-01, PNorm = 62.2647, GNorm = 7.3491, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.527238
Epoch 85
Validation binary_cross_entropy = 0.518482
Epoch 86
Validation binary_cross_entropy = 0.564762
Epoch 87
Validation binary_cross_entropy = 0.593962
Epoch 88
Validation binary_cross_entropy = 0.640645
Epoch 89
Loss = 4.1676e-01, PNorm = 62.6308, GNorm = 21.2302, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.679094
Epoch 90
Validation binary_cross_entropy = 0.738668
Epoch 91
Validation binary_cross_entropy = 0.701578
Epoch 92
Validation binary_cross_entropy = 0.687876
Epoch 93
Validation binary_cross_entropy = 0.814821
Epoch 94
Loss = 3.8760e-01, PNorm = 62.9675, GNorm = 15.8537, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.726864
Epoch 95
Validation binary_cross_entropy = 0.750259
Epoch 96
Validation binary_cross_entropy = 0.879491
Epoch 97
Validation binary_cross_entropy = 0.805625
Epoch 98
Validation binary_cross_entropy = 0.708828
Epoch 99
Loss = 3.1133e-01, PNorm = 63.3135, GNorm = 20.2556, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.721426
Epoch 100
Validation binary_cross_entropy = 0.919912
Epoch 101
Validation binary_cross_entropy = 0.876721
Epoch 102
Validation binary_cross_entropy = 0.846974
Epoch 103
Validation binary_cross_entropy = 0.828826
Epoch 104
Loss = 1.3237e-01, PNorm = 63.6219, GNorm = 7.7584, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.918775
Epoch 105
Validation binary_cross_entropy = 0.830957
Epoch 106
Validation binary_cross_entropy = 0.819259
Epoch 107
Validation binary_cross_entropy = 0.769069
Epoch 108
Validation binary_cross_entropy = 0.762505
Epoch 109
Loss = 2.7752e-01, PNorm = 63.8812, GNorm = 9.5209, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.809152
Epoch 110
Validation binary_cross_entropy = 0.909926
Epoch 111
Validation binary_cross_entropy = 0.879139
Epoch 112
Validation binary_cross_entropy = 0.997312
Epoch 113
Validation binary_cross_entropy = 0.856315
Epoch 114
Loss = 3.1078e-01, PNorm = 64.1615, GNorm = 13.0595, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.897856
Epoch 115
Validation binary_cross_entropy = 0.949831
Epoch 116
Validation binary_cross_entropy = 0.852371
Epoch 117
Validation binary_cross_entropy = 0.830087
Epoch 118
Validation binary_cross_entropy = 0.845482
Epoch 119
Loss = 1.4750e-01, PNorm = 64.4013, GNorm = 9.2130, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.828827
Epoch 120
Validation binary_cross_entropy = 0.830788
Epoch 121
Validation binary_cross_entropy = 0.843991
Epoch 122
Validation binary_cross_entropy = 0.877766
Epoch 123
Validation binary_cross_entropy = 0.868456
Epoch 124
Loss = 1.2187e-01, PNorm = 64.6051, GNorm = 3.9722, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.866230
Epoch 125
Validation binary_cross_entropy = 0.864839
Epoch 126
Validation binary_cross_entropy = 0.885239
Epoch 127
Validation binary_cross_entropy = 0.925345
Epoch 128
Validation binary_cross_entropy = 0.914638
Epoch 129
Loss = 5.3832e-02, PNorm = 64.7431, GNorm = 3.7597, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.919472
Epoch 130
Validation binary_cross_entropy = 0.937871
Epoch 131
Validation binary_cross_entropy = 0.948125
Epoch 132
Validation binary_cross_entropy = 0.978201
Epoch 133
Validation binary_cross_entropy = 1.014936
Epoch 134
Loss = 4.1295e-02, PNorm = 64.8795, GNorm = 1.3394, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.059932
Epoch 135
Validation binary_cross_entropy = 1.044178
Epoch 136
Validation binary_cross_entropy = 1.034352
Epoch 137
Validation binary_cross_entropy = 1.041279
Epoch 138
Validation binary_cross_entropy = 1.054992
Epoch 139
Loss = 1.9000e-01, PNorm = 65.0215, GNorm = 3.9960, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.039410
Epoch 140
Validation binary_cross_entropy = 1.080485
Epoch 141
Validation binary_cross_entropy = 1.099688
Epoch 142
Validation binary_cross_entropy = 1.056137
Epoch 143
Validation binary_cross_entropy = 0.993863
Epoch 144
Loss = 1.9195e-01, PNorm = 65.1946, GNorm = 13.5900, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.923534
Epoch 145
Validation binary_cross_entropy = 0.860830
Epoch 146
Validation binary_cross_entropy = 0.878003
Epoch 147
Validation binary_cross_entropy = 0.862009
Epoch 148
Validation binary_cross_entropy = 0.934670
Epoch 149
Loss = 5.8335e-02, PNorm = 65.4179, GNorm = 5.9915, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.040474
Epoch 150
Validation binary_cross_entropy = 1.031652
Epoch 151
Validation binary_cross_entropy = 0.904067
Epoch 152
Validation binary_cross_entropy = 0.840383
Epoch 153
Validation binary_cross_entropy = 0.862171
Epoch 154
Loss = 1.6703e-01, PNorm = 65.6226, GNorm = 13.3685, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.904245
Epoch 155
Validation binary_cross_entropy = 0.926793
Epoch 156
Validation binary_cross_entropy = 0.928909
Epoch 157
Validation binary_cross_entropy = 0.963746
Epoch 158
Validation binary_cross_entropy = 0.942878
Epoch 159
Loss = 3.4187e-02, PNorm = 65.7863, GNorm = 3.4026, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.988521
Epoch 160
Validation binary_cross_entropy = 1.004590
Epoch 161
Validation binary_cross_entropy = 1.000007
Epoch 162
Validation binary_cross_entropy = 1.021315
Epoch 163
Validation binary_cross_entropy = 1.051871
Epoch 164
Loss = 2.7410e-02, PNorm = 65.9226, GNorm = 1.1056, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.082950
Epoch 165
Validation binary_cross_entropy = 1.159269
Epoch 166
Validation binary_cross_entropy = 1.184270
Epoch 167
Validation binary_cross_entropy = 1.137526
Epoch 168
Validation binary_cross_entropy = 1.011386
Epoch 169
Loss = 7.7301e-02, PNorm = 66.0146, GNorm = 3.4354, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.993639
Epoch 170
Validation binary_cross_entropy = 0.994251
Epoch 171
Validation binary_cross_entropy = 1.001204
Epoch 172
Validation binary_cross_entropy = 1.029225
Epoch 173
Validation binary_cross_entropy = 1.039160
Epoch 174
Loss = 1.1214e-01, PNorm = 66.1117, GNorm = 4.5888, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.032740
Epoch 175
Validation binary_cross_entropy = 1.038175
Epoch 176
Validation binary_cross_entropy = 1.103906
Epoch 177
Validation binary_cross_entropy = 1.095872
Epoch 178
Validation binary_cross_entropy = 1.138690
Epoch 179
Loss = 2.8559e-02, PNorm = 66.2285, GNorm = 6.1766, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.172623
Epoch 180
Validation binary_cross_entropy = 1.243485
Epoch 181
Validation binary_cross_entropy = 1.302071
Epoch 182
Validation binary_cross_entropy = 1.336768
Epoch 183
Validation binary_cross_entropy = 1.373825
Epoch 184
Loss = 1.7740e-02, PNorm = 66.3634, GNorm = 1.0543, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.405815
Epoch 185
Validation binary_cross_entropy = 1.398384
Epoch 186
Validation binary_cross_entropy = 1.366024
Epoch 187
Validation binary_cross_entropy = 1.338729
Epoch 188
Validation binary_cross_entropy = 1.260313
Epoch 189
Loss = 1.1829e-02, PNorm = 66.4602, GNorm = 2.2744, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.178034
Epoch 190
Validation binary_cross_entropy = 1.135249
Epoch 191
Validation binary_cross_entropy = 1.138075
Epoch 192
Validation binary_cross_entropy = 1.133420
Epoch 193
Validation binary_cross_entropy = 1.141985
Epoch 194
Loss = 3.9522e-02, PNorm = 66.5599, GNorm = 2.0452, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.169133
Epoch 195
Validation binary_cross_entropy = 1.208680
Epoch 196
Validation binary_cross_entropy = 1.244427
Epoch 197
Validation binary_cross_entropy = 1.290602
Epoch 198
Validation binary_cross_entropy = 1.329361
Epoch 199
Loss = 2.9778e-02, PNorm = 66.6636, GNorm = 2.1783, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.345455
Epoch 200
Validation binary_cross_entropy = 1.420018
Epoch 201
Validation binary_cross_entropy = 1.475416
Epoch 202
Validation binary_cross_entropy = 1.487002
Epoch 203
Validation binary_cross_entropy = 1.481079
Epoch 204
Loss = 5.1833e-03, PNorm = 66.7556, GNorm = 0.5501, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.495786
Epoch 205
Validation binary_cross_entropy = 1.471203
Epoch 206
Validation binary_cross_entropy = 1.405075
Epoch 207
Validation binary_cross_entropy = 1.360646
Epoch 208
Validation binary_cross_entropy = 1.364310
Epoch 209
Loss = 2.0737e-02, PNorm = 66.8604, GNorm = 2.5489, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.369477
Epoch 210
Validation binary_cross_entropy = 1.382021
Epoch 211
Validation binary_cross_entropy = 1.391591
Epoch 212
Validation binary_cross_entropy = 1.376480
Epoch 213
Validation binary_cross_entropy = 1.420087
Epoch 214
Loss = 3.0725e-02, PNorm = 66.9715, GNorm = 2.9560, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.406674
Epoch 215
Validation binary_cross_entropy = 1.411558
Epoch 216
Validation binary_cross_entropy = 1.427525
Epoch 217
Validation binary_cross_entropy = 1.405689
Epoch 218
Validation binary_cross_entropy = 1.394956
Epoch 219
Loss = 2.4569e-02, PNorm = 67.0509, GNorm = 3.3818, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.387484
Epoch 220
Validation binary_cross_entropy = 1.360727
Epoch 221
Validation binary_cross_entropy = 1.334483
Epoch 222
Validation binary_cross_entropy = 1.289444
Epoch 223
Validation binary_cross_entropy = 1.275883
Epoch 224
Loss = 6.7181e-03, PNorm = 67.1720, GNorm = 1.5395, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.269677
Epoch 225
Validation binary_cross_entropy = 1.266067
Epoch 226
Validation binary_cross_entropy = 1.261839
Epoch 227
Validation binary_cross_entropy = 1.275533
Epoch 228
Validation binary_cross_entropy = 1.298109
Epoch 229
Loss = 4.0555e-02, PNorm = 67.2798, GNorm = 5.3159, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.259674
Epoch 230
Validation binary_cross_entropy = 1.259419
Epoch 231
Validation binary_cross_entropy = 1.279704
Epoch 232
Validation binary_cross_entropy = 1.353412
Epoch 233
Validation binary_cross_entropy = 1.331936
Epoch 234
Loss = 4.6578e-02, PNorm = 67.3639, GNorm = 1.1081, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.328159
Epoch 235
Validation binary_cross_entropy = 1.336943
Epoch 236
Validation binary_cross_entropy = 1.353908
Epoch 237
Validation binary_cross_entropy = 1.378293
Epoch 238
Validation binary_cross_entropy = 1.399647
Epoch 239
Loss = 2.5923e-02, PNorm = 67.4420, GNorm = 2.8414, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.423154
Epoch 240
Validation binary_cross_entropy = 1.473434
Epoch 241
Validation binary_cross_entropy = 1.529877
Epoch 242
Validation binary_cross_entropy = 1.554602
Epoch 243
Validation binary_cross_entropy = 1.558257
Epoch 244
Loss = 2.5357e-02, PNorm = 67.5029, GNorm = 2.8209, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.565927
Epoch 245
Validation binary_cross_entropy = 1.583014
Epoch 246
Validation binary_cross_entropy = 1.607949
Epoch 247
Validation binary_cross_entropy = 1.592661
Epoch 248
Validation binary_cross_entropy = 1.584270
Epoch 249
Loss = 4.1619e-04, PNorm = 67.5598, GNorm = 0.0274, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.575893
Epoch 250
Validation binary_cross_entropy = 1.572850
Epoch 251
Validation binary_cross_entropy = 1.605022
Epoch 252
Validation binary_cross_entropy = 1.599003
Epoch 253
Validation binary_cross_entropy = 1.560215
Epoch 254
Loss = 5.4750e-03, PNorm = 67.5975, GNorm = 0.0417, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.565256
Epoch 255
Validation binary_cross_entropy = 1.580574
Epoch 256
Validation binary_cross_entropy = 1.560483
Epoch 257
Validation binary_cross_entropy = 1.578424
Epoch 258
Validation binary_cross_entropy = 1.549069
Epoch 259
Loss = 1.7010e-02, PNorm = 67.6464, GNorm = 1.7715, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.483669
Epoch 260
Validation binary_cross_entropy = 1.471931
Epoch 261
Validation binary_cross_entropy = 1.460620
Epoch 262
Validation binary_cross_entropy = 1.463363
Epoch 263
Validation binary_cross_entropy = 1.545193
Epoch 264
Loss = 3.3587e-02, PNorm = 67.7127, GNorm = 1.2251, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.520121
Epoch 265
Validation binary_cross_entropy = 1.460169
Epoch 266
Validation binary_cross_entropy = 1.447942
Epoch 267
Validation binary_cross_entropy = 1.455898
Epoch 268
Validation binary_cross_entropy = 1.465676
Epoch 269
Loss = 8.1990e-03, PNorm = 67.7652, GNorm = 1.7350, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.482773
Epoch 270
Validation binary_cross_entropy = 1.494856
Epoch 271
Validation binary_cross_entropy = 1.512435
Epoch 272
Validation binary_cross_entropy = 1.521879
Epoch 273
Validation binary_cross_entropy = 1.522073
Epoch 274
Loss = 2.6720e-02, PNorm = 67.8062, GNorm = 2.9462, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.531001
Epoch 275
Validation binary_cross_entropy = 1.545245
Epoch 276
Validation binary_cross_entropy = 1.561258
Epoch 277
Validation binary_cross_entropy = 1.606111
Epoch 278
Validation binary_cross_entropy = 1.689520
Epoch 279
Loss = 3.9022e-03, PNorm = 67.8600, GNorm = 0.1788, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.813719
Epoch 280
Validation binary_cross_entropy = 1.877295
Epoch 281
Validation binary_cross_entropy = 1.881232
Epoch 282
Validation binary_cross_entropy = 1.892021
Epoch 283
Validation binary_cross_entropy = 1.864416
Epoch 284
Loss = 3.9061e-03, PNorm = 67.9141, GNorm = 0.1655, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.827255
Epoch 285
Validation binary_cross_entropy = 1.810036
Epoch 286
Validation binary_cross_entropy = 1.896765
Epoch 287
Validation binary_cross_entropy = 2.046175
Epoch 288
Validation binary_cross_entropy = 2.115627
Epoch 289
Loss = 2.0837e-02, PNorm = 67.9787, GNorm = 5.7434, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 2.102588
Epoch 290
Validation binary_cross_entropy = 1.885214
Epoch 291
Validation binary_cross_entropy = 1.885370
Epoch 292
Validation binary_cross_entropy = 1.828741
Epoch 293
Validation binary_cross_entropy = 1.917159
Epoch 294
Loss = 2.7875e-02, PNorm = 68.0617, GNorm = 4.5556, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 2.010735
Epoch 295
Validation binary_cross_entropy = 1.875113
Epoch 296
Validation binary_cross_entropy = 1.834448
Epoch 297
Validation binary_cross_entropy = 1.821709
Epoch 298
Validation binary_cross_entropy = 1.817571
Epoch 299
Loss = 6.8001e-03, PNorm = 68.1292, GNorm = 0.6080, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.847871
Model 0 best validation binary_cross_entropy = 0.301799 on epoch 53
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.223528
Ensemble test binary_cross_entropy = 0.223528
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.031441
Epoch 1
Validation binary_cross_entropy = 1.722067
Epoch 2
Validation binary_cross_entropy = 2.015496
Epoch 3
Validation binary_cross_entropy = 1.147266
Epoch 4
Loss = 1.8146e+00, PNorm = 57.7635, GNorm = 16.9101, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.576245
Epoch 5
Validation binary_cross_entropy = 1.552895
Epoch 6
Validation binary_cross_entropy = 1.521005
Epoch 7
Validation binary_cross_entropy = 0.706955
Epoch 8
Validation binary_cross_entropy = 0.475002
Epoch 9
Loss = 1.5243e+00, PNorm = 57.9715, GNorm = 26.2074, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.539534
Epoch 10
Validation binary_cross_entropy = 0.778923
Epoch 11
Validation binary_cross_entropy = 0.913055
Epoch 12
Validation binary_cross_entropy = 1.265080
Epoch 13
Validation binary_cross_entropy = 0.969400
Epoch 14
Loss = 1.5679e+00, PNorm = 58.2523, GNorm = 21.2607, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.539995
Epoch 15
Validation binary_cross_entropy = 0.590568
Epoch 16
Validation binary_cross_entropy = 0.736147
Epoch 17
Validation binary_cross_entropy = 0.839752
Epoch 18
Validation binary_cross_entropy = 0.841657
Epoch 19
Loss = 1.4748e+00, PNorm = 58.4438, GNorm = 18.7340, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.470356
Epoch 20
Validation binary_cross_entropy = 0.307252
Epoch 21
Validation binary_cross_entropy = 0.311025
Epoch 22
Validation binary_cross_entropy = 0.338816
Epoch 23
Validation binary_cross_entropy = 0.373042
Epoch 24
Loss = 1.1240e+00, PNorm = 58.5956, GNorm = 19.5511, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.412518
Epoch 25
Validation binary_cross_entropy = 0.521161
Epoch 26
Validation binary_cross_entropy = 0.437370
Epoch 27
Validation binary_cross_entropy = 0.440052
Epoch 28
Validation binary_cross_entropy = 0.425004
Epoch 29
Loss = 9.7945e-01, PNorm = 58.7547, GNorm = 15.4314, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.479700
Epoch 30
Validation binary_cross_entropy = 0.509637
Epoch 31
Validation binary_cross_entropy = 0.392651
Epoch 32
Validation binary_cross_entropy = 0.668061
Epoch 33
Validation binary_cross_entropy = 0.845693
Epoch 34
Loss = 9.9153e-01, PNorm = 58.9254, GNorm = 33.1907, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.527878
Epoch 35
Validation binary_cross_entropy = 0.320815
Epoch 36
Validation binary_cross_entropy = 0.325871
Epoch 37
Validation binary_cross_entropy = 0.412225
Epoch 38
Validation binary_cross_entropy = 0.579293
Epoch 39
Loss = 9.1208e-01, PNorm = 59.0753, GNorm = 22.6956, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.523064
Epoch 40
Validation binary_cross_entropy = 0.419940
Epoch 41
Validation binary_cross_entropy = 0.371390
Epoch 42
Validation binary_cross_entropy = 0.371793
Epoch 43
Validation binary_cross_entropy = 0.432469
Epoch 44
Loss = 8.2437e-01, PNorm = 59.2764, GNorm = 17.7745, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.565328
Epoch 45
Validation binary_cross_entropy = 0.563840
Epoch 46
Validation binary_cross_entropy = 0.406569
Epoch 47
Validation binary_cross_entropy = 0.407051
Epoch 48
Validation binary_cross_entropy = 0.488209
Epoch 49
Loss = 8.7617e-01, PNorm = 59.5033, GNorm = 21.1838, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.381793
Epoch 50
Validation binary_cross_entropy = 0.313780
Epoch 51
Validation binary_cross_entropy = 0.314565
Epoch 52
Validation binary_cross_entropy = 0.379023
Epoch 53
Validation binary_cross_entropy = 0.529356
Epoch 54
Loss = 6.7391e-01, PNorm = 59.7555, GNorm = 5.9523, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.434669
Epoch 55
Validation binary_cross_entropy = 0.368189
Epoch 56
Validation binary_cross_entropy = 0.414317
Epoch 57
Validation binary_cross_entropy = 0.423310
Epoch 58
Validation binary_cross_entropy = 0.363968
Epoch 59
Loss = 7.5150e-01, PNorm = 60.0295, GNorm = 6.9693, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.389885
Epoch 60
Validation binary_cross_entropy = 0.593688
Epoch 61
Validation binary_cross_entropy = 0.345022
Epoch 62
Validation binary_cross_entropy = 0.419782
Epoch 63
Validation binary_cross_entropy = 0.508556
Epoch 64
Loss = 6.4316e-01, PNorm = 60.3552, GNorm = 8.1736, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.373531
Epoch 65
Validation binary_cross_entropy = 0.382460
Epoch 66
Validation binary_cross_entropy = 0.369604
Epoch 67
Validation binary_cross_entropy = 0.476445
Epoch 68
Validation binary_cross_entropy = 0.563023
Epoch 69
Loss = 5.0865e-01, PNorm = 60.7281, GNorm = 11.0829, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.410465
Epoch 70
Validation binary_cross_entropy = 0.392000
Epoch 71
Validation binary_cross_entropy = 0.475465
Epoch 72
Validation binary_cross_entropy = 0.383715
Epoch 73
Validation binary_cross_entropy = 0.380457
Epoch 74
Loss = 3.8079e-01, PNorm = 61.1089, GNorm = 14.2858, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.482495
Epoch 75
Validation binary_cross_entropy = 0.476382
Epoch 76
Validation binary_cross_entropy = 0.470122
Epoch 77
Validation binary_cross_entropy = 0.550484
Epoch 78
Validation binary_cross_entropy = 0.551334
Epoch 79
Loss = 4.3188e-01, PNorm = 61.5084, GNorm = 17.7911, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.747584
Epoch 80
Validation binary_cross_entropy = 0.481714
Epoch 81
Validation binary_cross_entropy = 0.448289
Epoch 82
Validation binary_cross_entropy = 0.683589
Epoch 83
Validation binary_cross_entropy = 0.546252
Epoch 84
Loss = 4.1938e-01, PNorm = 61.9055, GNorm = 10.6731, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.441938
Epoch 85
Validation binary_cross_entropy = 0.442139
Epoch 86
Validation binary_cross_entropy = 0.501116
Epoch 87
Validation binary_cross_entropy = 0.434235
Epoch 88
Validation binary_cross_entropy = 0.454567
Epoch 89
Loss = 4.5469e-01, PNorm = 62.2755, GNorm = 24.6018, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.495873
Epoch 90
Validation binary_cross_entropy = 0.491140
Epoch 91
Validation binary_cross_entropy = 0.421334
Epoch 92
Validation binary_cross_entropy = 0.436227
Epoch 93
Validation binary_cross_entropy = 0.471398
Epoch 94
Loss = 3.7275e-01, PNorm = 62.6264, GNorm = 11.0361, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.654054
Epoch 95
Validation binary_cross_entropy = 0.526740
Epoch 96
Validation binary_cross_entropy = 0.514720
Epoch 97
Validation binary_cross_entropy = 0.537730
Epoch 98
Validation binary_cross_entropy = 0.544989
Epoch 99
Loss = 2.6203e-01, PNorm = 62.9904, GNorm = 11.0729, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.571848
Epoch 100
Validation binary_cross_entropy = 0.617678
Epoch 101
Validation binary_cross_entropy = 0.672064
Epoch 102
Validation binary_cross_entropy = 0.559951
Epoch 103
Validation binary_cross_entropy = 0.564040
Epoch 104
Loss = 1.4296e-01, PNorm = 63.2931, GNorm = 7.0274, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.748684
Epoch 105
Validation binary_cross_entropy = 0.584193
Epoch 106
Validation binary_cross_entropy = 0.594498
Epoch 107
Validation binary_cross_entropy = 0.560594
Epoch 108
Validation binary_cross_entropy = 0.622783
Epoch 109
Loss = 1.9679e-01, PNorm = 63.5639, GNorm = 19.2674, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.583662
Epoch 110
Validation binary_cross_entropy = 0.548863
Epoch 111
Validation binary_cross_entropy = 0.588241
Epoch 112
Validation binary_cross_entropy = 0.581668
Epoch 113
Validation binary_cross_entropy = 0.667902
Epoch 114
Loss = 1.8204e-01, PNorm = 63.8581, GNorm = 14.5196, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.654877
Epoch 115
Validation binary_cross_entropy = 0.677605
Epoch 116
Validation binary_cross_entropy = 0.672855
Epoch 117
Validation binary_cross_entropy = 0.691936
Epoch 118
Validation binary_cross_entropy = 0.712409
Epoch 119
Loss = 1.1872e-01, PNorm = 64.1473, GNorm = 3.6671, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.739439
Epoch 120
Validation binary_cross_entropy = 0.744457
Epoch 121
Validation binary_cross_entropy = 0.730245
Epoch 122
Validation binary_cross_entropy = 0.763669
Epoch 123
Validation binary_cross_entropy = 0.714797
Epoch 124
Loss = 1.3284e-01, PNorm = 64.3887, GNorm = 4.9958, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.706993
Epoch 125
Validation binary_cross_entropy = 0.719405
Epoch 126
Validation binary_cross_entropy = 0.741292
Epoch 127
Validation binary_cross_entropy = 0.727406
Epoch 128
Validation binary_cross_entropy = 0.689218
Epoch 129
Loss = 6.6423e-02, PNorm = 64.6195, GNorm = 3.8574, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.814491
Epoch 130
Validation binary_cross_entropy = 0.729566
Epoch 131
Validation binary_cross_entropy = 0.755824
Epoch 132
Validation binary_cross_entropy = 0.729459
Epoch 133
Validation binary_cross_entropy = 0.759524
Epoch 134
Loss = 1.6855e-01, PNorm = 64.8896, GNorm = 4.8386, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.778597
Epoch 135
Validation binary_cross_entropy = 0.793838
Epoch 136
Validation binary_cross_entropy = 0.947298
Epoch 137
Validation binary_cross_entropy = 0.817275
Epoch 138
Validation binary_cross_entropy = 0.742120
Epoch 139
Loss = 1.1478e-01, PNorm = 65.1285, GNorm = 7.6383, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.687852
Epoch 140
Validation binary_cross_entropy = 0.732983
Epoch 141
Validation binary_cross_entropy = 0.925166
Epoch 142
Validation binary_cross_entropy = 0.804071
Epoch 143
Validation binary_cross_entropy = 0.809584
Epoch 144
Loss = 2.5458e-01, PNorm = 65.3605, GNorm = 4.8890, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.842366
Epoch 145
Validation binary_cross_entropy = 0.883207
Epoch 146
Validation binary_cross_entropy = 0.854076
Epoch 147
Validation binary_cross_entropy = 0.861831
Epoch 148
Validation binary_cross_entropy = 0.910193
Epoch 149
Loss = 1.8316e-01, PNorm = 65.5735, GNorm = 3.8644, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.880772
Epoch 150
Validation binary_cross_entropy = 0.830398
Epoch 151
Validation binary_cross_entropy = 0.789198
Epoch 152
Validation binary_cross_entropy = 0.703092
Epoch 153
Validation binary_cross_entropy = 0.682249
Epoch 154
Loss = 9.8657e-02, PNorm = 65.7809, GNorm = 4.0513, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.675400
Epoch 155
Validation binary_cross_entropy = 0.689255
Epoch 156
Validation binary_cross_entropy = 0.714819
Epoch 157
Validation binary_cross_entropy = 0.858931
Epoch 158
Validation binary_cross_entropy = 0.893921
Epoch 159
Loss = 2.0364e-01, PNorm = 65.9888, GNorm = 15.1866, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.814051
Epoch 160
Validation binary_cross_entropy = 0.805093
Epoch 161
Validation binary_cross_entropy = 0.807231
Epoch 162
Validation binary_cross_entropy = 0.852104
Epoch 163
Validation binary_cross_entropy = 0.937460
Epoch 164
Loss = 3.9977e-02, PNorm = 66.2121, GNorm = 4.8264, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.040976
Epoch 165
Validation binary_cross_entropy = 1.121835
Epoch 166
Validation binary_cross_entropy = 1.141758
Epoch 167
Validation binary_cross_entropy = 1.100024
Epoch 168
Validation binary_cross_entropy = 1.044894
Epoch 169
Loss = 7.6078e-02, PNorm = 66.4131, GNorm = 6.8986, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.014298
Epoch 170
Validation binary_cross_entropy = 0.971001
Epoch 171
Validation binary_cross_entropy = 0.957927
Epoch 172
Validation binary_cross_entropy = 0.962422
Epoch 173
Validation binary_cross_entropy = 0.969161
Epoch 174
Loss = 8.0242e-02, PNorm = 66.5857, GNorm = 9.8638, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.974420
Epoch 175
Validation binary_cross_entropy = 0.992088
Epoch 176
Validation binary_cross_entropy = 1.027417
Epoch 177
Validation binary_cross_entropy = 1.050628
Epoch 178
Validation binary_cross_entropy = 0.983614
Epoch 179
Loss = 5.6982e-02, PNorm = 66.7273, GNorm = 7.0081, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.027473
Epoch 180
Validation binary_cross_entropy = 1.012857
Epoch 181
Validation binary_cross_entropy = 1.019434
Epoch 182
Validation binary_cross_entropy = 1.041785
Epoch 183
Validation binary_cross_entropy = 1.078422
Epoch 184
Loss = 8.4464e-02, PNorm = 66.8823, GNorm = 5.9930, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.109947
Epoch 185
Validation binary_cross_entropy = 1.153597
Epoch 186
Validation binary_cross_entropy = 1.202213
Epoch 187
Validation binary_cross_entropy = 1.247408
Epoch 188
Validation binary_cross_entropy = 1.288590
Epoch 189
Loss = 3.9901e-02, PNorm = 67.0183, GNorm = 3.7030, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.270187
Epoch 190
Validation binary_cross_entropy = 1.233410
Epoch 191
Validation binary_cross_entropy = 1.191434
Epoch 192
Validation binary_cross_entropy = 1.177649
Epoch 193
Validation binary_cross_entropy = 1.165893
Epoch 194
Loss = 3.2245e-02, PNorm = 67.1405, GNorm = 2.7697, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.161530
Epoch 195
Validation binary_cross_entropy = 1.169442
Epoch 196
Validation binary_cross_entropy = 1.272793
Epoch 197
Validation binary_cross_entropy = 1.506720
Epoch 198
Validation binary_cross_entropy = 1.438720
Epoch 199
Loss = 1.0252e-01, PNorm = 67.2601, GNorm = 7.2886, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.420429
Epoch 200
Validation binary_cross_entropy = 1.365759
Epoch 201
Validation binary_cross_entropy = 1.315982
Epoch 202
Validation binary_cross_entropy = 1.274306
Epoch 203
Validation binary_cross_entropy = 1.259047
Epoch 204
Loss = 4.0617e-02, PNorm = 67.3815, GNorm = 4.9534, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.264022
Epoch 205
Validation binary_cross_entropy = 1.177067
Epoch 206
Validation binary_cross_entropy = 1.171456
Epoch 207
Validation binary_cross_entropy = 1.159959
Epoch 208
Validation binary_cross_entropy = 1.136108
Epoch 209
Loss = 1.9635e-02, PNorm = 67.5116, GNorm = 3.0904, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.138673
Epoch 210
Validation binary_cross_entropy = 1.143653
Epoch 211
Validation binary_cross_entropy = 1.171347
Epoch 212
Validation binary_cross_entropy = 1.221631
Epoch 213
Validation binary_cross_entropy = 1.287117
Epoch 214
Loss = 4.8570e-02, PNorm = 67.6146, GNorm = 11.3334, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.298845
Epoch 215
Validation binary_cross_entropy = 1.311342
Epoch 216
Validation binary_cross_entropy = 1.323855
Epoch 217
Validation binary_cross_entropy = 1.332435
Epoch 218
Validation binary_cross_entropy = 1.349798
Epoch 219
Loss = 5.0452e-02, PNorm = 67.6982, GNorm = 2.1303, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.319919
Epoch 220
Validation binary_cross_entropy = 1.324065
Epoch 221
Validation binary_cross_entropy = 1.333095
Epoch 222
Validation binary_cross_entropy = 1.365480
Epoch 223
Validation binary_cross_entropy = 1.417888
Epoch 224
Loss = 7.7841e-02, PNorm = 67.7844, GNorm = 6.4985, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.503922
Epoch 225
Validation binary_cross_entropy = 1.485384
Epoch 226
Validation binary_cross_entropy = 1.363625
Epoch 227
Validation binary_cross_entropy = 1.350379
Epoch 228
Validation binary_cross_entropy = 1.363352
Epoch 229
Loss = 4.0725e-02, PNorm = 67.8778, GNorm = 3.6716, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.421292
Epoch 230
Validation binary_cross_entropy = 1.462271
Epoch 231
Validation binary_cross_entropy = 1.363281
Epoch 232
Validation binary_cross_entropy = 1.304447
Epoch 233
Validation binary_cross_entropy = 1.336658
Epoch 234
Loss = 4.8109e-02, PNorm = 67.9545, GNorm = 5.5030, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.352980
Epoch 235
Validation binary_cross_entropy = 1.313961
Epoch 236
Validation binary_cross_entropy = 1.303300
Epoch 237
Validation binary_cross_entropy = 1.297977
Epoch 238
Validation binary_cross_entropy = 1.267654
Epoch 239
Loss = 1.4495e-02, PNorm = 68.0489, GNorm = 2.1476, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.247921
Epoch 240
Validation binary_cross_entropy = 1.252900
Epoch 241
Validation binary_cross_entropy = 1.261182
Epoch 242
Validation binary_cross_entropy = 1.288684
Epoch 243
Validation binary_cross_entropy = 1.311886
Epoch 244
Loss = 6.5261e-02, PNorm = 68.1248, GNorm = 10.5179, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.322321
Epoch 245
Validation binary_cross_entropy = 1.341958
Epoch 246
Validation binary_cross_entropy = 1.305639
Epoch 247
Validation binary_cross_entropy = 1.322478
Epoch 248
Validation binary_cross_entropy = 1.351701
Epoch 249
Loss = 3.0810e-02, PNorm = 68.2130, GNorm = 4.7864, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.386255
Epoch 250
Validation binary_cross_entropy = 1.411533
Epoch 251
Validation binary_cross_entropy = 1.422503
Epoch 252
Validation binary_cross_entropy = 1.445934
Epoch 253
Validation binary_cross_entropy = 1.444928
Epoch 254
Loss = 3.3377e-02, PNorm = 68.2995, GNorm = 1.2009, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.444175
Epoch 255
Validation binary_cross_entropy = 1.477750
Epoch 256
Validation binary_cross_entropy = 1.482852
Epoch 257
Validation binary_cross_entropy = 1.490982
Epoch 258
Validation binary_cross_entropy = 1.550167
Epoch 259
Loss = 5.8853e-02, PNorm = 68.3765, GNorm = 0.1818, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.494552
Epoch 260
Validation binary_cross_entropy = 1.464901
Epoch 261
Validation binary_cross_entropy = 1.427221
Epoch 262
Validation binary_cross_entropy = 1.409731
Epoch 263
Validation binary_cross_entropy = 1.415561
Epoch 264
Loss = 3.2390e-02, PNorm = 68.4187, GNorm = 3.9344, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.421238
Epoch 265
Validation binary_cross_entropy = 1.428648
Epoch 266
Validation binary_cross_entropy = 1.424601
Epoch 267
Validation binary_cross_entropy = 1.430776
Epoch 268
Validation binary_cross_entropy = 1.435298
Epoch 269
Loss = 5.9520e-02, PNorm = 68.4647, GNorm = 1.8364, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.431021
Epoch 270
Validation binary_cross_entropy = 1.438111
Epoch 271
Validation binary_cross_entropy = 1.444867
Epoch 272
Validation binary_cross_entropy = 1.453893
Epoch 273
Validation binary_cross_entropy = 1.465736
Epoch 274
Loss = 1.5696e-02, PNorm = 68.5178, GNorm = 2.5924, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.500242
Epoch 275
Validation binary_cross_entropy = 1.558010
Epoch 276
Validation binary_cross_entropy = 1.581119
Epoch 277
Validation binary_cross_entropy = 1.587088
Epoch 278
Validation binary_cross_entropy = 1.607258
Epoch 279
Loss = 2.0285e-03, PNorm = 68.5768, GNorm = 0.1806, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.634541
Epoch 280
Validation binary_cross_entropy = 1.634285
Epoch 281
Validation binary_cross_entropy = 1.664894
Epoch 282
Validation binary_cross_entropy = 1.680091
Epoch 283
Validation binary_cross_entropy = 1.591369
Epoch 284
Loss = 4.2998e-02, PNorm = 68.6323, GNorm = 4.4634, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.519541
Epoch 285
Validation binary_cross_entropy = 1.479601
Epoch 286
Validation binary_cross_entropy = 1.461562
Epoch 287
Validation binary_cross_entropy = 1.429823
Epoch 288
Validation binary_cross_entropy = 1.442580
Epoch 289
Loss = 1.4122e-02, PNorm = 68.7003, GNorm = 0.8416, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.441047
Epoch 290
Validation binary_cross_entropy = 1.430482
Epoch 291
Validation binary_cross_entropy = 1.450338
Epoch 292
Validation binary_cross_entropy = 1.494186
Epoch 293
Validation binary_cross_entropy = 1.522472
Epoch 294
Loss = 3.8588e-02, PNorm = 68.7595, GNorm = 0.2172, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.543375
Epoch 295
Validation binary_cross_entropy = 1.528796
Epoch 296
Validation binary_cross_entropy = 1.545237
Epoch 297
Validation binary_cross_entropy = 1.502354
Epoch 298
Validation binary_cross_entropy = 1.522314
Epoch 299
Loss = 4.6335e-03, PNorm = 68.8596, GNorm = 0.1317, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.577137
Model 0 best validation binary_cross_entropy = 0.307252 on epoch 20
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.250411
Ensemble test binary_cross_entropy = 0.250411
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.916868
Epoch 1
Validation binary_cross_entropy = 1.340962
Epoch 2
Validation binary_cross_entropy = 0.954824
Epoch 3
Validation binary_cross_entropy = 1.347826
Epoch 4
Loss = 2.3624e+00, PNorm = 57.7664, GNorm = 30.0666, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 1.605031
Epoch 5
Validation binary_cross_entropy = 0.681825
Epoch 6
Validation binary_cross_entropy = 0.496868
Epoch 7
Validation binary_cross_entropy = 0.779211
Epoch 8
Validation binary_cross_entropy = 1.095291
Epoch 9
Loss = 1.7526e+00, PNorm = 57.9920, GNorm = 24.6834, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.613152
Epoch 10
Validation binary_cross_entropy = 0.451839
Epoch 11
Validation binary_cross_entropy = 0.595099
Epoch 12
Validation binary_cross_entropy = 1.025294
Epoch 13
Validation binary_cross_entropy = 0.959420
Epoch 14
Loss = 1.6124e+00, PNorm = 58.2516, GNorm = 17.1060, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.623622
Epoch 15
Validation binary_cross_entropy = 0.429127
Epoch 16
Validation binary_cross_entropy = 0.423373
Epoch 17
Validation binary_cross_entropy = 0.615848
Epoch 18
Validation binary_cross_entropy = 0.623368
Epoch 19
Loss = 1.1943e+00, PNorm = 58.4658, GNorm = 24.1224, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.565161
Epoch 20
Validation binary_cross_entropy = 0.349243
Epoch 21
Validation binary_cross_entropy = 0.374331
Epoch 22
Validation binary_cross_entropy = 0.864857
Epoch 23
Validation binary_cross_entropy = 0.883300
Epoch 24
Loss = 1.5394e+00, PNorm = 58.6335, GNorm = 16.0840, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.459375
Epoch 25
Validation binary_cross_entropy = 0.329000
Epoch 26
Validation binary_cross_entropy = 0.311913
Epoch 27
Validation binary_cross_entropy = 0.361393
Epoch 28
Validation binary_cross_entropy = 0.443166
Epoch 29
Loss = 7.2111e-01, PNorm = 58.7965, GNorm = 11.3150, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.535890
Epoch 30
Validation binary_cross_entropy = 0.633196
Epoch 31
Validation binary_cross_entropy = 0.476277
Epoch 32
Validation binary_cross_entropy = 0.358286
Epoch 33
Validation binary_cross_entropy = 0.331388
Epoch 34
Loss = 7.8352e-01, PNorm = 58.9611, GNorm = 17.1459, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.421089
Epoch 35
Validation binary_cross_entropy = 0.697108
Epoch 36
Validation binary_cross_entropy = 0.930023
Epoch 37
Validation binary_cross_entropy = 0.482273
Epoch 38
Validation binary_cross_entropy = 0.335098
Epoch 39
Loss = 9.0244e-01, PNorm = 59.1686, GNorm = 25.3684, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.419628
Epoch 40
Validation binary_cross_entropy = 0.611529
Epoch 41
Validation binary_cross_entropy = 0.567518
Epoch 42
Validation binary_cross_entropy = 0.484757
Epoch 43
Validation binary_cross_entropy = 0.414190
Epoch 44
Loss = 8.0579e-01, PNorm = 59.3876, GNorm = 12.0831, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.397628
Epoch 45
Validation binary_cross_entropy = 0.499260
Epoch 46
Validation binary_cross_entropy = 0.614307
Epoch 47
Validation binary_cross_entropy = 0.512700
Epoch 48
Validation binary_cross_entropy = 0.354635
Epoch 49
Loss = 6.2412e-01, PNorm = 59.6116, GNorm = 8.2740, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.324889
Epoch 50
Validation binary_cross_entropy = 0.409840
Epoch 51
Validation binary_cross_entropy = 0.518314
Epoch 52
Validation binary_cross_entropy = 0.408962
Epoch 53
Validation binary_cross_entropy = 0.386181
Epoch 54
Loss = 7.5628e-01, PNorm = 59.8698, GNorm = 19.6138, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.433285
Epoch 55
Validation binary_cross_entropy = 0.472162
Epoch 56
Validation binary_cross_entropy = 0.429467
Epoch 57
Validation binary_cross_entropy = 0.388543
Epoch 58
Validation binary_cross_entropy = 0.435054
Epoch 59
Loss = 6.5161e-01, PNorm = 60.1870, GNorm = 24.3405, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.413484
Epoch 60
Validation binary_cross_entropy = 0.346104
Epoch 61
Validation binary_cross_entropy = 0.330767
Epoch 62
Validation binary_cross_entropy = 0.574785
Epoch 63
Validation binary_cross_entropy = 0.609996
Epoch 64
Loss = 5.6980e-01, PNorm = 60.5182, GNorm = 7.2009, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.390757
Epoch 65
Validation binary_cross_entropy = 0.346262
Epoch 66
Validation binary_cross_entropy = 0.472946
Epoch 67
Validation binary_cross_entropy = 0.509708
Epoch 68
Validation binary_cross_entropy = 0.364678
Epoch 69
Loss = 6.7770e-01, PNorm = 60.8120, GNorm = 13.6765, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.334949
Epoch 70
Validation binary_cross_entropy = 0.393555
Epoch 71
Validation binary_cross_entropy = 0.504291
Epoch 72
Validation binary_cross_entropy = 0.501974
Epoch 73
Validation binary_cross_entropy = 0.415907
Epoch 74
Loss = 3.9045e-01, PNorm = 61.1152, GNorm = 10.2061, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.388143
Epoch 75
Validation binary_cross_entropy = 0.406784
Epoch 76
Validation binary_cross_entropy = 0.570469
Epoch 77
Validation binary_cross_entropy = 0.592551
Epoch 78
Validation binary_cross_entropy = 0.440364
Epoch 79
Loss = 5.2034e-01, PNorm = 61.4527, GNorm = 21.9676, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.448958
Epoch 80
Validation binary_cross_entropy = 0.506348
Epoch 81
Validation binary_cross_entropy = 0.480198
Epoch 82
Validation binary_cross_entropy = 0.485364
Epoch 83
Validation binary_cross_entropy = 0.398635
Epoch 84
Loss = 3.1878e-01, PNorm = 61.8097, GNorm = 10.9509, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.387922
Epoch 85
Validation binary_cross_entropy = 0.403498
Epoch 86
Validation binary_cross_entropy = 0.511147
Epoch 87
Validation binary_cross_entropy = 0.538762
Epoch 88
Validation binary_cross_entropy = 0.491859
Epoch 89
Loss = 3.1654e-01, PNorm = 62.1825, GNorm = 9.1727, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.452258
Epoch 90
Validation binary_cross_entropy = 0.475633
Epoch 91
Validation binary_cross_entropy = 0.519386
Epoch 92
Validation binary_cross_entropy = 0.585714
Epoch 93
Validation binary_cross_entropy = 0.548527
Epoch 94
Loss = 3.6928e-01, PNorm = 62.5430, GNorm = 7.9343, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.562020
Epoch 95
Validation binary_cross_entropy = 0.679213
Epoch 96
Validation binary_cross_entropy = 0.479956
Epoch 97
Validation binary_cross_entropy = 0.449619
Epoch 98
Validation binary_cross_entropy = 0.427393
Epoch 99
Loss = 3.1760e-01, PNorm = 62.9069, GNorm = 13.2334, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.736346
Epoch 100
Validation binary_cross_entropy = 0.480560
Epoch 101
Validation binary_cross_entropy = 0.491262
Epoch 102
Validation binary_cross_entropy = 0.466165
Epoch 103
Validation binary_cross_entropy = 0.514281
Epoch 104
Loss = 2.3752e-01, PNorm = 63.2775, GNorm = 13.0991, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.586187
Epoch 105
Validation binary_cross_entropy = 0.467516
Epoch 106
Validation binary_cross_entropy = 0.497543
Epoch 107
Validation binary_cross_entropy = 0.460723
Epoch 108
Validation binary_cross_entropy = 0.467631
Epoch 109
Loss = 1.7567e-01, PNorm = 63.6039, GNorm = 10.6879, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.509410
Epoch 110
Validation binary_cross_entropy = 0.530544
Epoch 111
Validation binary_cross_entropy = 0.561696
Epoch 112
Validation binary_cross_entropy = 0.489106
Epoch 113
Validation binary_cross_entropy = 0.599720
Epoch 114
Loss = 4.5522e-01, PNorm = 63.8890, GNorm = 7.4068, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.496346
Epoch 115
Validation binary_cross_entropy = 0.542787
Epoch 116
Validation binary_cross_entropy = 0.539545
Epoch 117
Validation binary_cross_entropy = 0.608184
Epoch 118
Validation binary_cross_entropy = 0.652143
Epoch 119
Loss = 3.8815e-01, PNorm = 64.1863, GNorm = 6.9296, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.713011
Epoch 120
Validation binary_cross_entropy = 0.641367
Epoch 121
Validation binary_cross_entropy = 0.639190
Epoch 122
Validation binary_cross_entropy = 0.743769
Epoch 123
Validation binary_cross_entropy = 0.665510
Epoch 124
Loss = 1.7356e-01, PNorm = 64.4696, GNorm = 8.1615, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.656349
Epoch 125
Validation binary_cross_entropy = 0.803387
Epoch 126
Validation binary_cross_entropy = 0.798941
Epoch 127
Validation binary_cross_entropy = 0.720035
Epoch 128
Validation binary_cross_entropy = 0.630211
Epoch 129
Loss = 6.2540e-02, PNorm = 64.7354, GNorm = 2.8640, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.582355
Epoch 130
Validation binary_cross_entropy = 0.538504
Epoch 131
Validation binary_cross_entropy = 0.511349
Epoch 132
Validation binary_cross_entropy = 0.527238
Epoch 133
Validation binary_cross_entropy = 0.522697
Epoch 134
Loss = 8.6731e-02, PNorm = 64.9439, GNorm = 2.5072, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.537407
Epoch 135
Validation binary_cross_entropy = 0.568303
Epoch 136
Validation binary_cross_entropy = 0.564351
Epoch 137
Validation binary_cross_entropy = 0.601790
Epoch 138
Validation binary_cross_entropy = 0.681344
Epoch 139
Loss = 9.7736e-02, PNorm = 65.1173, GNorm = 4.0426, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.694306
Epoch 140
Validation binary_cross_entropy = 0.784116
Epoch 141
Validation binary_cross_entropy = 0.901929
Epoch 142
Validation binary_cross_entropy = 0.820064
Epoch 143
Validation binary_cross_entropy = 0.870746
Epoch 144
Loss = 1.9607e-01, PNorm = 65.2921, GNorm = 14.5754, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.764786
Epoch 145
Validation binary_cross_entropy = 0.708770
Epoch 146
Validation binary_cross_entropy = 0.712479
Epoch 147
Validation binary_cross_entropy = 0.673180
Epoch 148
Validation binary_cross_entropy = 0.684202
Epoch 149
Loss = 1.5747e-01, PNorm = 65.4897, GNorm = 17.0743, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.690964
Epoch 150
Validation binary_cross_entropy = 0.695941
Epoch 151
Validation binary_cross_entropy = 0.724312
Epoch 152
Validation binary_cross_entropy = 0.763794
Epoch 153
Validation binary_cross_entropy = 0.811909
Epoch 154
Loss = 5.6364e-02, PNorm = 65.6794, GNorm = 3.7583, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.844959
Epoch 155
Validation binary_cross_entropy = 0.855619
Epoch 156
Validation binary_cross_entropy = 0.875326
Epoch 157
Validation binary_cross_entropy = 0.891676
Epoch 158
Validation binary_cross_entropy = 0.873282
Epoch 159
Loss = 3.8528e-02, PNorm = 65.8286, GNorm = 6.7871, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.887478
Epoch 160
Validation binary_cross_entropy = 0.896904
Epoch 161
Validation binary_cross_entropy = 0.919818
Epoch 162
Validation binary_cross_entropy = 0.892279
Epoch 163
Validation binary_cross_entropy = 0.864048
Epoch 164
Loss = 6.3718e-02, PNorm = 65.9624, GNorm = 8.8418, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.876107
Epoch 165
Validation binary_cross_entropy = 0.903424
Epoch 166
Validation binary_cross_entropy = 0.846564
Epoch 167
Validation binary_cross_entropy = 0.864583
Epoch 168
Validation binary_cross_entropy = 0.913689
Epoch 169
Loss = 1.3308e-01, PNorm = 66.1333, GNorm = 10.5298, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.977200
Epoch 170
Validation binary_cross_entropy = 1.038415
Epoch 171
Validation binary_cross_entropy = 1.143704
Epoch 172
Validation binary_cross_entropy = 1.181466
Epoch 173
Validation binary_cross_entropy = 1.164434
Epoch 174
Loss = 3.5379e-02, PNorm = 66.3004, GNorm = 0.3711, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.255863
Epoch 175
Validation binary_cross_entropy = 1.130577
Epoch 176
Validation binary_cross_entropy = 1.193735
Epoch 177
Validation binary_cross_entropy = 1.146386
Epoch 178
Validation binary_cross_entropy = 1.086954
Epoch 179
Loss = 1.2615e-01, PNorm = 66.4563, GNorm = 1.3292, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.672590
Epoch 180
Validation binary_cross_entropy = 1.049094
Epoch 181
Validation binary_cross_entropy = 1.041430
Epoch 182
Validation binary_cross_entropy = 1.075408
Epoch 183
Validation binary_cross_entropy = 0.998672
Epoch 184
Loss = 9.6870e-03, PNorm = 66.5909, GNorm = 0.2272, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.014781
Epoch 185
Validation binary_cross_entropy = 1.088995
Epoch 186
Validation binary_cross_entropy = 0.950197
Epoch 187
Validation binary_cross_entropy = 0.982994
Epoch 188
Validation binary_cross_entropy = 1.027514
Epoch 189
Loss = 1.0505e-01, PNorm = 66.7445, GNorm = 12.6025, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.951015
Epoch 190
Validation binary_cross_entropy = 0.889364
Epoch 191
Validation binary_cross_entropy = 0.945357
Epoch 192
Validation binary_cross_entropy = 1.000951
Epoch 193
Validation binary_cross_entropy = 0.931277
Epoch 194
Loss = 3.4077e-02, PNorm = 66.9113, GNorm = 2.0269, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.959732
Epoch 195
Validation binary_cross_entropy = 0.991189
Epoch 196
Validation binary_cross_entropy = 0.970378
Epoch 197
Validation binary_cross_entropy = 1.000076
Epoch 198
Validation binary_cross_entropy = 1.065598
Epoch 199
Loss = 9.5637e-02, PNorm = 67.0554, GNorm = 6.6664, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.157671
Epoch 200
Validation binary_cross_entropy = 1.313804
Epoch 201
Validation binary_cross_entropy = 1.316490
Epoch 202
Validation binary_cross_entropy = 1.129646
Epoch 203
Validation binary_cross_entropy = 0.974771
Epoch 204
Loss = 1.4192e-02, PNorm = 67.1641, GNorm = 4.1692, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.940678
Epoch 205
Validation binary_cross_entropy = 0.916795
Epoch 206
Validation binary_cross_entropy = 1.019146
Epoch 207
Validation binary_cross_entropy = 0.971993
Epoch 208
Validation binary_cross_entropy = 0.892390
Epoch 209
Loss = 2.1828e-02, PNorm = 67.3148, GNorm = 1.6592, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.874558
Epoch 210
Validation binary_cross_entropy = 0.875538
Epoch 211
Validation binary_cross_entropy = 0.891502
Epoch 212
Validation binary_cross_entropy = 0.963760
Epoch 213
Validation binary_cross_entropy = 1.010231
Epoch 214
Loss = 5.1037e-02, PNorm = 67.4555, GNorm = 2.8954, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.002505
Epoch 215
Validation binary_cross_entropy = 1.102494
Epoch 216
Validation binary_cross_entropy = 1.233402
Epoch 217
Validation binary_cross_entropy = 1.200098
Epoch 218
Validation binary_cross_entropy = 1.168643
Epoch 219
Loss = 3.3838e-02, PNorm = 67.6114, GNorm = 1.5949, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.172987
Epoch 220
Validation binary_cross_entropy = 1.167796
Epoch 221
Validation binary_cross_entropy = 1.116377
Epoch 222
Validation binary_cross_entropy = 1.038445
Epoch 223
Validation binary_cross_entropy = 0.992524
Epoch 224
Loss = 3.4314e-02, PNorm = 67.7792, GNorm = 9.0467, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.971839
Epoch 225
Validation binary_cross_entropy = 1.043562
Epoch 226
Validation binary_cross_entropy = 1.072311
Epoch 227
Validation binary_cross_entropy = 1.018024
Epoch 228
Validation binary_cross_entropy = 1.059572
Epoch 229
Loss = 1.1643e-01, PNorm = 67.9232, GNorm = 10.4823, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.071582
Epoch 230
Validation binary_cross_entropy = 1.098819
Epoch 231
Validation binary_cross_entropy = 1.218507
Epoch 232
Validation binary_cross_entropy = 1.195827
Epoch 233
Validation binary_cross_entropy = 1.082976
Epoch 234
Loss = 3.7154e-03, PNorm = 68.0923, GNorm = 0.2948, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.101763
Epoch 235
Validation binary_cross_entropy = 1.079886
Epoch 236
Validation binary_cross_entropy = 1.034198
Epoch 237
Validation binary_cross_entropy = 1.003052
Epoch 238
Validation binary_cross_entropy = 1.039615
Epoch 239
Loss = 4.0571e-02, PNorm = 68.2404, GNorm = 2.0021, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.066123
Epoch 240
Validation binary_cross_entropy = 1.074721
Epoch 241
Validation binary_cross_entropy = 1.076762
Epoch 242
Validation binary_cross_entropy = 1.119604
Epoch 243
Validation binary_cross_entropy = 1.166737
Epoch 244
Loss = 1.3946e-02, PNorm = 68.3402, GNorm = 0.0558, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.206601
Epoch 245
Validation binary_cross_entropy = 1.247500
Epoch 246
Validation binary_cross_entropy = 1.295463
Epoch 247
Validation binary_cross_entropy = 1.315405
Epoch 248
Validation binary_cross_entropy = 1.311902
Epoch 249
Loss = 2.4790e-02, PNorm = 68.4316, GNorm = 3.4777, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.330398
Epoch 250
Validation binary_cross_entropy = 1.353897
Epoch 251
Validation binary_cross_entropy = 1.364834
Epoch 252
Validation binary_cross_entropy = 1.358953
Epoch 253
Validation binary_cross_entropy = 1.364764
Epoch 254
Loss = 5.0967e-02, PNorm = 68.5172, GNorm = 0.4589, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.380417
Epoch 255
Validation binary_cross_entropy = 1.401003
Epoch 256
Validation binary_cross_entropy = 1.376761
Epoch 257
Validation binary_cross_entropy = 1.351098
Epoch 258
Validation binary_cross_entropy = 1.343530
Epoch 259
Loss = 3.2036e-02, PNorm = 68.5700, GNorm = 5.9165, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.350605
Epoch 260
Validation binary_cross_entropy = 1.372490
Epoch 261
Validation binary_cross_entropy = 1.403766
Epoch 262
Validation binary_cross_entropy = 1.386671
Epoch 263
Validation binary_cross_entropy = 1.310926
Epoch 264
Loss = 1.2801e-02, PNorm = 68.6199, GNorm = 5.6023, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.247110
Epoch 265
Validation binary_cross_entropy = 1.221740
Epoch 266
Validation binary_cross_entropy = 1.206411
Epoch 267
Validation binary_cross_entropy = 1.198190
Epoch 268
Validation binary_cross_entropy = 1.209132
Epoch 269
Loss = 7.4548e-02, PNorm = 68.6958, GNorm = 0.7189, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.290061
Epoch 270
Validation binary_cross_entropy = 1.371929
Epoch 271
Validation binary_cross_entropy = 1.383530
Epoch 272
Validation binary_cross_entropy = 1.366980
Epoch 273
Validation binary_cross_entropy = 1.352548
Epoch 274
Loss = 8.9292e-03, PNorm = 68.7621, GNorm = 1.7558, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.361849
Epoch 275
Validation binary_cross_entropy = 1.368959
Epoch 276
Validation binary_cross_entropy = 1.374184
Epoch 277
Validation binary_cross_entropy = 1.380479
Epoch 278
Validation binary_cross_entropy = 1.411858
Epoch 279
Loss = 2.2401e-02, PNorm = 68.8201, GNorm = 2.5214, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.450396
Epoch 280
Validation binary_cross_entropy = 1.469554
Epoch 281
Validation binary_cross_entropy = 1.447312
Epoch 282
Validation binary_cross_entropy = 1.463853
Epoch 283
Validation binary_cross_entropy = 1.528424
Epoch 284
Loss = 3.5020e-03, PNorm = 68.8671, GNorm = 0.3770, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.596759
Epoch 285
Validation binary_cross_entropy = 1.598987
Epoch 286
Validation binary_cross_entropy = 1.576061
Epoch 287
Validation binary_cross_entropy = 1.570174
Epoch 288
Validation binary_cross_entropy = 1.478752
Epoch 289
Loss = 6.7966e-03, PNorm = 68.9045, GNorm = 1.0689, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.455888
Epoch 290
Validation binary_cross_entropy = 1.390709
Epoch 291
Validation binary_cross_entropy = 1.356371
Epoch 292
Validation binary_cross_entropy = 1.365637
Epoch 293
Validation binary_cross_entropy = 1.401141
Epoch 294
Loss = 6.9999e-02, PNorm = 69.0116, GNorm = 2.8326, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.364825
Epoch 295
Validation binary_cross_entropy = 1.376583
Epoch 296
Validation binary_cross_entropy = 1.418508
Epoch 297
Validation binary_cross_entropy = 1.448938
Epoch 298
Validation binary_cross_entropy = 1.456814
Epoch 299
Loss = 1.6972e-02, PNorm = 69.1154, GNorm = 0.1122, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.497907
Model 0 best validation binary_cross_entropy = 0.311913 on epoch 26
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.258313
Ensemble test binary_cross_entropy = 0.258313
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1500, bias=False)
        (W_h): Linear(in_features=1500, out_features=1500, bias=False)
        (W_o): Linear(in_features=1633, out_features=1500, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 4,923,301
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.204365
Epoch 1
Validation binary_cross_entropy = 1.742229
Epoch 2
Validation binary_cross_entropy = 1.243726
Epoch 3
Validation binary_cross_entropy = 0.954935
Epoch 4
Loss = 1.6830e+00, PNorm = 57.7626, GNorm = 20.8333, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.811478
Epoch 5
Validation binary_cross_entropy = 0.609963
Epoch 6
Validation binary_cross_entropy = 0.929889
Epoch 7
Validation binary_cross_entropy = 0.501617
Epoch 8
Validation binary_cross_entropy = 1.272343
Epoch 9
Loss = 1.5542e+00, PNorm = 57.9701, GNorm = 46.5744, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.804070
Epoch 10
Validation binary_cross_entropy = 0.434007
Epoch 11
Validation binary_cross_entropy = 0.508288
Epoch 12
Validation binary_cross_entropy = 0.685541
Epoch 13
Validation binary_cross_entropy = 0.725768
Epoch 14
Loss = 1.3260e+00, PNorm = 58.2182, GNorm = 18.2460, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.629986
Epoch 15
Validation binary_cross_entropy = 0.391182
Epoch 16
Validation binary_cross_entropy = 0.544444
Epoch 17
Validation binary_cross_entropy = 0.932611
Epoch 18
Validation binary_cross_entropy = 0.572892
Epoch 19
Loss = 1.2047e+00, PNorm = 58.4275, GNorm = 16.4124, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.382811
Epoch 20
Validation binary_cross_entropy = 0.363882
Epoch 21
Validation binary_cross_entropy = 0.482469
Epoch 22
Validation binary_cross_entropy = 0.505920
Epoch 23
Validation binary_cross_entropy = 0.500100
Epoch 24
Loss = 9.6128e-01, PNorm = 58.5987, GNorm = 24.1640, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.460163
Epoch 25
Validation binary_cross_entropy = 0.544295
Epoch 26
Validation binary_cross_entropy = 0.437047
Epoch 27
Validation binary_cross_entropy = 0.338096
Epoch 28
Validation binary_cross_entropy = 0.642028
Epoch 29
Loss = 1.1414e+00, PNorm = 58.7812, GNorm = 15.0025, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.517202
Epoch 30
Validation binary_cross_entropy = 0.332619
Epoch 31
Validation binary_cross_entropy = 0.313488
Epoch 32
Validation binary_cross_entropy = 0.447003
Epoch 33
Validation binary_cross_entropy = 0.720389
Epoch 34
Loss = 9.8028e-01, PNorm = 58.9569, GNorm = 30.4854, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.582928
Epoch 35
Validation binary_cross_entropy = 0.357091
Epoch 36
Validation binary_cross_entropy = 0.312501
Epoch 37
Validation binary_cross_entropy = 0.348119
Epoch 38
Validation binary_cross_entropy = 0.473812
Epoch 39
Loss = 9.5597e-01, PNorm = 59.1575, GNorm = 20.9704, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.492256
Epoch 40
Validation binary_cross_entropy = 0.379707
Epoch 41
Validation binary_cross_entropy = 0.361000
Epoch 42
Validation binary_cross_entropy = 0.456946
Epoch 43
Validation binary_cross_entropy = 0.489216
Epoch 44
Loss = 9.7425e-01, PNorm = 59.4182, GNorm = 11.0295, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.321358
Epoch 45
Validation binary_cross_entropy = 0.433629
Epoch 46
Validation binary_cross_entropy = 0.677463
Epoch 47
Validation binary_cross_entropy = 0.340705
Epoch 48
Validation binary_cross_entropy = 0.293036
Epoch 49
Loss = 7.7437e-01, PNorm = 59.6721, GNorm = 20.6775, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.302544
Epoch 50
Validation binary_cross_entropy = 0.454168
Epoch 51
Validation binary_cross_entropy = 0.614740
Epoch 52
Validation binary_cross_entropy = 0.531750
Epoch 53
Validation binary_cross_entropy = 0.386814
Epoch 54
Loss = 8.1499e-01, PNorm = 59.9607, GNorm = 19.7481, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.577439
Epoch 55
Validation binary_cross_entropy = 0.452110
Epoch 56
Validation binary_cross_entropy = 0.365002
Epoch 57
Validation binary_cross_entropy = 0.385563
Epoch 58
Validation binary_cross_entropy = 0.627703
Epoch 59
Loss = 6.2719e-01, PNorm = 60.2775, GNorm = 18.5054, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.584304
Epoch 60
Validation binary_cross_entropy = 0.375003
Epoch 61
Validation binary_cross_entropy = 0.414929
Epoch 62
Validation binary_cross_entropy = 0.523501
Epoch 63
Validation binary_cross_entropy = 0.466642
Epoch 64
Loss = 5.0263e-01, PNorm = 60.6412, GNorm = 16.7488, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.496699
Epoch 65
Validation binary_cross_entropy = 0.579654
Epoch 66
Validation binary_cross_entropy = 0.566875
Epoch 67
Validation binary_cross_entropy = 0.949626
Epoch 68
Validation binary_cross_entropy = 0.547829
Epoch 69
Loss = 5.6537e-01, PNorm = 61.0012, GNorm = 20.6880, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.455856
Epoch 70
Validation binary_cross_entropy = 0.696534
Epoch 71
Validation binary_cross_entropy = 0.385634
Epoch 72
Validation binary_cross_entropy = 0.348047
Epoch 73
Validation binary_cross_entropy = 0.356208
Epoch 74
Loss = 5.3284e-01, PNorm = 61.3411, GNorm = 7.6767, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.517069
Epoch 75
Validation binary_cross_entropy = 0.526409
Epoch 76
Validation binary_cross_entropy = 0.407197
Epoch 77
Validation binary_cross_entropy = 0.418709
Epoch 78
Validation binary_cross_entropy = 0.580273
Epoch 79
Loss = 5.6066e-01, PNorm = 61.6863, GNorm = 28.0860, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.491025
Epoch 80
Validation binary_cross_entropy = 0.453838
Epoch 81
Validation binary_cross_entropy = 0.495868
Epoch 82
Validation binary_cross_entropy = 0.488484
Epoch 83
Validation binary_cross_entropy = 0.534474
Epoch 84
Loss = 1.6248e-01, PNorm = 62.0125, GNorm = 13.5524, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.497364
Epoch 85
Validation binary_cross_entropy = 0.514571
Epoch 86
Validation binary_cross_entropy = 0.666487
Epoch 87
Validation binary_cross_entropy = 0.550294
Epoch 88
Validation binary_cross_entropy = 0.548186
Epoch 89
Loss = 3.0965e-01, PNorm = 62.3144, GNorm = 19.8044, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.562753
Epoch 90
Validation binary_cross_entropy = 0.587463
Epoch 91
Validation binary_cross_entropy = 0.576826
Epoch 92
Validation binary_cross_entropy = 0.577782
Epoch 93
Validation binary_cross_entropy = 0.623719
Epoch 94
Loss = 2.5380e-01, PNorm = 62.6469, GNorm = 6.8633, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.542051
Epoch 95
Validation binary_cross_entropy = 0.568544
Epoch 96
Validation binary_cross_entropy = 0.582555
Epoch 97
Validation binary_cross_entropy = 0.683247
Epoch 98
Validation binary_cross_entropy = 0.609136
Epoch 99
Loss = 2.7550e-01, PNorm = 62.9599, GNorm = 23.6027, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.620912
Epoch 100
Validation binary_cross_entropy = 0.654614
Epoch 101
Validation binary_cross_entropy = 0.798349
Epoch 102
Validation binary_cross_entropy = 0.529576
Epoch 103
Validation binary_cross_entropy = 0.537352
Epoch 104
Loss = 3.3061e-01, PNorm = 63.2460, GNorm = 9.1273, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.498714
Epoch 105
Validation binary_cross_entropy = 0.572106
Epoch 106
Validation binary_cross_entropy = 0.629787
Epoch 107
Validation binary_cross_entropy = 0.584382
Epoch 108
Validation binary_cross_entropy = 0.650774
Epoch 109
Loss = 1.8602e-01, PNorm = 63.5306, GNorm = 22.5859, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.561974
Epoch 110
Validation binary_cross_entropy = 0.644634
Epoch 111
Validation binary_cross_entropy = 0.580941
Epoch 112
Validation binary_cross_entropy = 0.605666
Epoch 113
Validation binary_cross_entropy = 0.644547
Epoch 114
Loss = 2.1805e-01, PNorm = 63.8369, GNorm = 25.0685, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.771850
Epoch 115
Validation binary_cross_entropy = 0.564577
Epoch 116
Validation binary_cross_entropy = 0.520164
Epoch 117
Validation binary_cross_entropy = 0.483020
Epoch 118
Validation binary_cross_entropy = 0.657407
Epoch 119
Loss = 2.2870e-01, PNorm = 64.1621, GNorm = 25.4546, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.662673
Epoch 120
Validation binary_cross_entropy = 0.632878
Epoch 121
Validation binary_cross_entropy = 0.594741
Epoch 122
Validation binary_cross_entropy = 0.617190
Epoch 123
Validation binary_cross_entropy = 0.642711
Epoch 124
Loss = 2.4028e-01, PNorm = 64.4638, GNorm = 10.7436, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.598020
Epoch 125
Validation binary_cross_entropy = 0.603913
Epoch 126
Validation binary_cross_entropy = 0.624267
Epoch 127
Validation binary_cross_entropy = 0.631126
Epoch 128
Validation binary_cross_entropy = 0.612712
Epoch 129
Loss = 1.2245e-01, PNorm = 64.7223, GNorm = 8.0809, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.593142
Epoch 130
Validation binary_cross_entropy = 0.612528
Epoch 131
Validation binary_cross_entropy = 0.617692
Epoch 132
Validation binary_cross_entropy = 0.630283
Epoch 133
Validation binary_cross_entropy = 0.645856
Epoch 134
Loss = 3.4487e-02, PNorm = 64.9567, GNorm = 2.7460, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.685134
Epoch 135
Validation binary_cross_entropy = 0.751092
Epoch 136
Validation binary_cross_entropy = 0.808727
Epoch 137
Validation binary_cross_entropy = 0.817410
Epoch 138
Validation binary_cross_entropy = 0.815407
Epoch 139
Loss = 1.0160e-01, PNorm = 65.1444, GNorm = 4.2882, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.801858
Epoch 140
Validation binary_cross_entropy = 0.819228
Epoch 141
Validation binary_cross_entropy = 0.777541
Epoch 142
Validation binary_cross_entropy = 0.804631
Epoch 143
Validation binary_cross_entropy = 0.812579
Epoch 144
Loss = 5.7461e-02, PNorm = 65.3310, GNorm = 2.9702, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.860160
Epoch 145
Validation binary_cross_entropy = 0.846496
Epoch 146
Validation binary_cross_entropy = 0.808443
Epoch 147
Validation binary_cross_entropy = 0.826784
Epoch 148
Validation binary_cross_entropy = 0.775833
Epoch 149
Loss = 8.5554e-02, PNorm = 65.5059, GNorm = 5.2550, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.792493
Epoch 150
Validation binary_cross_entropy = 0.762699
Epoch 151
Validation binary_cross_entropy = 0.762621
Epoch 152
Validation binary_cross_entropy = 0.795872
Epoch 153
Validation binary_cross_entropy = 0.748457
Epoch 154
Loss = 8.7369e-02, PNorm = 65.7003, GNorm = 5.3340, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.756349
Epoch 155
Validation binary_cross_entropy = 0.815202
Epoch 156
Validation binary_cross_entropy = 0.849653
Epoch 157
Validation binary_cross_entropy = 0.901368
Epoch 158
Validation binary_cross_entropy = 0.973515
Epoch 159
Loss = 3.1775e-02, PNorm = 65.8645, GNorm = 1.8038, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.026494
Epoch 160
Validation binary_cross_entropy = 1.055262
Epoch 161
Validation binary_cross_entropy = 1.010964
Epoch 162
Validation binary_cross_entropy = 0.891851
Epoch 163
Validation binary_cross_entropy = 0.857554
Epoch 164
Loss = 5.2338e-02, PNorm = 66.0348, GNorm = 3.2976, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.849832
Epoch 165
Validation binary_cross_entropy = 0.881520
Epoch 166
Validation binary_cross_entropy = 0.826478
Epoch 167
Validation binary_cross_entropy = 0.813537
Epoch 168
Validation binary_cross_entropy = 0.820075
Epoch 169
Loss = 9.8571e-02, PNorm = 66.1843, GNorm = 4.3727, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.846156
Epoch 170
Validation binary_cross_entropy = 0.919596
Epoch 171
Validation binary_cross_entropy = 0.885692
Epoch 172
Validation binary_cross_entropy = 0.793689
Epoch 173
Validation binary_cross_entropy = 0.724428
Epoch 174
Loss = 8.1772e-02, PNorm = 66.3429, GNorm = 4.9866, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.721067
Epoch 175
Validation binary_cross_entropy = 0.744824
Epoch 176
Validation binary_cross_entropy = 0.807235
Epoch 177
Validation binary_cross_entropy = 0.896086
Epoch 178
Validation binary_cross_entropy = 0.956706
Epoch 179
Loss = 1.5349e-01, PNorm = 66.5194, GNorm = 2.7704, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.794531
Epoch 180
Validation binary_cross_entropy = 0.846559
Epoch 181
Validation binary_cross_entropy = 0.924406
Epoch 182
Validation binary_cross_entropy = 0.895684
Epoch 183
Validation binary_cross_entropy = 0.885864
Epoch 184
Loss = 9.7606e-02, PNorm = 66.6830, GNorm = 8.0911, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.897690
Epoch 185
Validation binary_cross_entropy = 0.892752
Epoch 186
Validation binary_cross_entropy = 0.873632
Epoch 187
Validation binary_cross_entropy = 0.832718
Epoch 188
Validation binary_cross_entropy = 0.891392
Epoch 189
Loss = 1.4289e-01, PNorm = 66.8559, GNorm = 18.1322, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.963683
Epoch 190
Validation binary_cross_entropy = 0.921487
Epoch 191
Validation binary_cross_entropy = 0.938090
Epoch 192
Validation binary_cross_entropy = 0.962015
Epoch 193
Validation binary_cross_entropy = 0.992223
Epoch 194
Loss = 2.7747e-02, PNorm = 67.0314, GNorm = 3.8652, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.039744
Epoch 195
Validation binary_cross_entropy = 1.114322
Epoch 196
Validation binary_cross_entropy = 1.049566
Epoch 197
Validation binary_cross_entropy = 1.048098
Epoch 198
Validation binary_cross_entropy = 1.027643
Epoch 199
Loss = 9.1823e-02, PNorm = 67.2024, GNorm = 4.4249, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.901304
Epoch 200
Validation binary_cross_entropy = 0.914635
Epoch 201
Validation binary_cross_entropy = 0.895398
Epoch 202
Validation binary_cross_entropy = 0.814230
Epoch 203
Validation binary_cross_entropy = 0.736173
Epoch 204
Loss = 1.6016e-01, PNorm = 67.3606, GNorm = 8.2588, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.732242
Epoch 205
Validation binary_cross_entropy = 0.765408
Epoch 206
Validation binary_cross_entropy = 0.841762
Epoch 207
Validation binary_cross_entropy = 0.890304
Epoch 208
Validation binary_cross_entropy = 0.858355
Epoch 209
Loss = 1.8426e-02, PNorm = 67.4912, GNorm = 0.7019, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.833805
Epoch 210
Validation binary_cross_entropy = 0.897361
Epoch 211
Validation binary_cross_entropy = 1.013394
Epoch 212
Validation binary_cross_entropy = 1.188728
Epoch 213
Validation binary_cross_entropy = 1.185007
Epoch 214
Loss = 3.7847e-02, PNorm = 67.6208, GNorm = 0.6871, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.076079
Epoch 215
Validation binary_cross_entropy = 1.047799
Epoch 216
Validation binary_cross_entropy = 1.064999
Epoch 217
Validation binary_cross_entropy = 1.086383
Epoch 218
Validation binary_cross_entropy = 1.083689
Epoch 219
Loss = 1.0129e-01, PNorm = 67.7022, GNorm = 3.5872, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.071299
Epoch 220
Validation binary_cross_entropy = 1.088615
Epoch 221
Validation binary_cross_entropy = 1.116315
Epoch 222
Validation binary_cross_entropy = 1.154931
Epoch 223
Validation binary_cross_entropy = 1.192847
Epoch 224
Loss = 2.7969e-02, PNorm = 67.7811, GNorm = 0.5006, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.239699
Epoch 225
Validation binary_cross_entropy = 1.273874
Epoch 226
Validation binary_cross_entropy = 1.264312
Epoch 227
Validation binary_cross_entropy = 1.251490
Epoch 228
Validation binary_cross_entropy = 1.276279
Epoch 229
Loss = 8.8396e-03, PNorm = 67.8774, GNorm = 0.9730, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.322202
Epoch 230
Validation binary_cross_entropy = 1.335781
Epoch 231
Validation binary_cross_entropy = 1.365323
Epoch 232
Validation binary_cross_entropy = 1.350071
Epoch 233
Validation binary_cross_entropy = 1.319256
Epoch 234
Loss = 5.7430e-03, PNorm = 67.9723, GNorm = 0.5683, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.322394
Epoch 235
Validation binary_cross_entropy = 1.327722
Epoch 236
Validation binary_cross_entropy = 1.347421
Epoch 237
Validation binary_cross_entropy = 1.329388
Epoch 238
Validation binary_cross_entropy = 1.322736
Epoch 239
Loss = 2.6730e-02, PNorm = 68.0427, GNorm = 2.5108, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.355582
Epoch 240
Validation binary_cross_entropy = 1.364139
Epoch 241
Validation binary_cross_entropy = 1.381127
Epoch 242
Validation binary_cross_entropy = 1.351223
Epoch 243
Validation binary_cross_entropy = 1.364267
Epoch 244
Loss = 1.3727e-02, PNorm = 68.1202, GNorm = 2.3445, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.373295
Epoch 245
Validation binary_cross_entropy = 1.359787
Epoch 246
Validation binary_cross_entropy = 1.351484
Epoch 247
Validation binary_cross_entropy = 1.349544
Epoch 248
Validation binary_cross_entropy = 1.371646
Epoch 249
Loss = 1.0753e-02, PNorm = 68.1842, GNorm = 1.2272, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.424570
Epoch 250
Validation binary_cross_entropy = 1.456838
Epoch 251
Validation binary_cross_entropy = 1.450547
Epoch 252
Validation binary_cross_entropy = 1.432561
Epoch 253
Validation binary_cross_entropy = 1.412509
Epoch 254
Loss = 9.7883e-03, PNorm = 68.2275, GNorm = 1.7186, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.417426
Epoch 255
Validation binary_cross_entropy = 1.437707
Epoch 256
Validation binary_cross_entropy = 1.474439
Epoch 257
Validation binary_cross_entropy = 1.520626
Epoch 258
Validation binary_cross_entropy = 1.556632
Epoch 259
Loss = 9.5742e-03, PNorm = 68.2687, GNorm = 5.4452, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.558822
Epoch 260
Validation binary_cross_entropy = 1.539501
Epoch 261
Validation binary_cross_entropy = 1.566348
Epoch 262
Validation binary_cross_entropy = 1.610891
Epoch 263
Validation binary_cross_entropy = 1.641136
Epoch 264
Loss = 2.0260e-02, PNorm = 68.3055, GNorm = 4.8384, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.560778
Epoch 265
Validation binary_cross_entropy = 1.476626
Epoch 266
Validation binary_cross_entropy = 1.474385
Epoch 267
Validation binary_cross_entropy = 1.487043
Epoch 268
Validation binary_cross_entropy = 1.494718
Epoch 269
Loss = 1.9119e-02, PNorm = 68.3527, GNorm = 4.2818, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.497861
Epoch 270
Validation binary_cross_entropy = 1.487465
Epoch 271
Validation binary_cross_entropy = 1.496256
Epoch 272
Validation binary_cross_entropy = 1.505026
Epoch 273
Validation binary_cross_entropy = 1.514131
Epoch 274
Loss = 4.6044e-02, PNorm = 68.3951, GNorm = 9.5881, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.508718
Epoch 275
Validation binary_cross_entropy = 1.497971
Epoch 276
Validation binary_cross_entropy = 1.481288
Epoch 277
Validation binary_cross_entropy = 1.467319
Epoch 278
Validation binary_cross_entropy = 1.463755
Epoch 279
Loss = 7.8391e-03, PNorm = 68.4278, GNorm = 1.2221, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.470000
Epoch 280
Validation binary_cross_entropy = 1.486169
Epoch 281
Validation binary_cross_entropy = 1.513531
Epoch 282
Validation binary_cross_entropy = 1.620013
Epoch 283
Validation binary_cross_entropy = 1.757417
Epoch 284
Loss = 5.7052e-02, PNorm = 68.4809, GNorm = 13.1696, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.660408
Epoch 285
Validation binary_cross_entropy = 1.633097
Epoch 286
Validation binary_cross_entropy = 1.636031
Epoch 287
Validation binary_cross_entropy = 1.826747
Epoch 288
Validation binary_cross_entropy = 2.027857
Epoch 289
Loss = 1.0776e-01, PNorm = 68.5926, GNorm = 1.9334, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.370302
Epoch 290
Validation binary_cross_entropy = 1.825222
Epoch 291
Validation binary_cross_entropy = 1.061832
Epoch 292
Validation binary_cross_entropy = 1.099978
Epoch 293
Validation binary_cross_entropy = 1.057304
Epoch 294
Loss = 6.0679e-02, PNorm = 68.8398, GNorm = 0.5856, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.904739
Epoch 295
Validation binary_cross_entropy = 0.917873
Epoch 296
Validation binary_cross_entropy = 0.949426
Epoch 297
Validation binary_cross_entropy = 0.923588
Epoch 298
Validation binary_cross_entropy = 0.958974
Epoch 299
Loss = 1.5379e-02, PNorm = 69.0467, GNorm = 1.6900, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.017357
Model 0 best validation binary_cross_entropy = 0.293036 on epoch 48
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.219444
Ensemble test binary_cross_entropy = 0.219444
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.263407
	Seed 1 ==> test binary_cross_entropy = 0.242477
	Seed 2 ==> test binary_cross_entropy = 0.228500
	Seed 3 ==> test binary_cross_entropy = 0.267992
	Seed 4 ==> test binary_cross_entropy = 0.265076
	Seed 5 ==> test binary_cross_entropy = 0.217307
	Seed 6 ==> test binary_cross_entropy = 0.223528
	Seed 7 ==> test binary_cross_entropy = 0.250411
	Seed 8 ==> test binary_cross_entropy = 0.258313
	Seed 9 ==> test binary_cross_entropy = 0.219444
Overall test binary_cross_entropy = 0.243645 +/- 0.019022
Elapsed time = 0:04:44
